Plans for E2-Productions.com to turn into a personal cloud?

Alert! Alert! Buzzword! Yes, before we start, let’s clear up with what I mean when I say "personal cloud". A personal cloud is a web-accessable system which centralises the function of common web 2.0 services, which may or may not be social. For those that aren’t familiar with this jargon, web 2.0 services are those such as web email clients like GMail, photo sharing and management sites like Flickr, online radios like Last.FM, and even blogs just like this one. So your personal cloud is a system on your very own website, with a web interface for your very own emails, PIM (calendar, notes, todos), images, music, etc. Note that the social attribute is optional. Clouds do not have to necessarily have automatic synchronisation, nor does it have to have the ability to easily share your data with the public.

A little history first. E2-Productions.com used to be the center of attention – thriving with the latest adolescent community fads such as animation, art and music "portals". It later saw the rise of the Blender Model Repository, a personal portfolio, several forums (trendy, weren’t they?), and finally ended with the death of the original animation portal. thinkMoult then emerged somewhere on the Blogspot blogosphere and went on to become a moderately-hacked install of WordPress on the E2 server. E2-Productions had become, and still is now, a dead site.

I’ve been toying with the idea of turning E2-Productions into a personal cloud for quite some time. It did actually occur at one point. Even though a PHP developer myself, it would take too long to create my very own cloud to implement existing free scripts. In the end I had created a network of individual PHP scripts giving me a web based RSS reader, filebin, imagebin, and a proxy. With the helpful addition of several rsync scripts and sshfs, it was usable and offered all the functionality. However of course it was very hacked together and didn’t offer the sort of integration I wanted.

Today I decided to look into other cloud services. One currently in development is ownCloud by Frank Karlitschek, the guy resonsible behind the openDesktop websites. Unfortunately the results were disappointing. Taking into account that it’s still under construction (and therefore incomplete and buggy), like the openDesktop websites themselves, ownCloud is unfortunately yet another developer service that underestimates usability. Whilst it had some nifty features in the works, it’s priorities were skewed away from the rightful mentality that design maketh a website, not the function. Further probing into the code revealed some serious problems with the structure of the coding that didn’t look very well thought out. Needless to say ownCloud is not for me.

The other famous personal cloud is the open-source EyeOS. This one goes all the way, completely replicating the desktop interface in the browser. Again, most of the design of EyeOS 1.x, their stable version, approaches the design of the system as a desktop interface. The canvas of a webpage is not suited for a desktop interface. They both have strengths and weaknesses and unfortunately most of these uncanny designs don’t play to the webbrowser’s strengths. Despite the overblown interface (which excusably is amended quite a bit in their 2.0 unstable version) it’s quite featureful and its extensibility in terms of developing it yourself is quite attractive. However the lag which accompanies such client-side interactive bloat (and server-side too!) doesn’t exactly make it the most practical of choices. It’s definitely worth keeping an eye on, though (excuse the pun).

Further searching yielded an exemplary system called Tonido. Unfortunately due to its proprietary nature it’s not for me either. However it does provide a fine example of the potential of a well executed cloud service. This motivated me to reconsider creating a cloud. I began with considering the basic user objectives for the web interface:

  • Ability to dump a file online with a unique private URL, and easily share it via public URL (or a single obfuscated URL).
  • Private browsing of my files with support for subdirectories.
  • Browsing of images will be represented in userfriendly thumbnail form.
  • Browsing of PIM data (vcard, ical) will be parsed and displayed in an appropriate format.
  • Web-based uploader and/or form of synchronisation technique.

You see the "cloud" I had described (in terms of its most basic user-side functionality) as such is pretty much just a smart web-based filebrowser with mini-webapp additions for (mainly) PIM-data. Our latest newcomer to the fad, Ubuntu One, actually provides these needs very well – leaving the browser to do what it does best, and the rest to the desktop. However it falls short in a vital area – proprietariness. This isn’t a question of evangelism, it’s more of one of the simple requirements of a personal cloud. If we analyse the 5 basic user-side needs to its roots, we get a shorter list of what actually makes a personal cloud, personal:

  • Fine control of private, limited, and public files.
  • Convenience – data should be searchable by tags, and there should be no limit on the filestructure or methods of access.
  • Timelessness – data should not be locked into any vendor.

The first issue is tackled quite well in existing cloud providers, with probably the best implementation (in my opinion) being Ubuntu One. Convenience is another easily satisfied need, with wonderful tools like rsync, sshfs, and version control software (though again, most providers lock you into their own system, and convenience ends once you leave it). However the key feature that in my eyes hasn’t yet been solved by any provider is timelessness. Any proprietary client or syncing software is instantly disqualified due to dependency on the vendor. Now, even if the software was completely open and extensible, many so called personal clouds are simply connecting to existing external services. Whilst consistent with the definition of the personal cloud, the service centralises in what is hoped to be seen as the path of least resistance – that is, only for people actually already using those services. What services am I talking about? Oh, things like Flickr, Google, Facebook and Twitter. Social is a hot topic, but not for everybody. Services don’t even have to be online – dependency on, for example Tomboy Notes or Evolution in Ubuntu One qualifies as a dependency and thus means the service is not timeless.

So how is this overcome? Not easily, for sure. I want to propose that the ideal personal cloud be one that focuses simply on file management and synchronisation. It should stop there – the actual display of files and searching of files should be handled by plugins. Plugins can decide how to properly format an image gallery. Plugins can decide how to display PIM-data. These plugins should be accomodating for the most timeless format ever, plaintext, as well as industry standard formats. The user then only applies the plugins that fits their exact workflow, if necessary writing their own for interpreting their own files. This satisfies the 3 criteria of a personal cloud.

I’m still coming up with a few plans and extra ideas on how it’ll be – but meanwhile I’ve got exams to get over. If anybody knows something similar that exists, let me know, otherwise this’ll be a fun project after WIPUP reaches stable.


The state of vendor lock-in on handheld services?

In this day and age it seems as though the word smartphone has replaced (or at least become synonymous with) the traditional phones we grew up with. These devices try to tackle the usual on-the-go services: PIM, messaging, casual browsing, multimedia and social networking. However with this is also an attempt to lock users into proprietary services, say for example, Flickr. I’ve been wondering for quite some time which mobile OS actually fares better on this, with the choices being the iPhone, Blackberry, Windows Mobile, Symbian, and WebOS (that’s Palm Pre).

This is best explained through example, so let’s take Mr Hip and Trendy who are well versed in navigating Facebook for their friends, Flickr or Picasa for their latest photo albums, Last.FM for the music, iTunes for their personal collection, YouTube for cats doing funny things and Twitter for trying to up their cool. Now let’s take away the Hip and Trendy part of all that and leave us with somebody who wants to do things their own way.

We have a neatly categorised library of music and video files on our home computer, and a few re-encoded video files on a remote server specifically made for mobile viewing. None of this iTunes schrwap. We run a shoutcast service for streaming, easily accessible through dyndns. Our latest photo albums, ebook library, and latest LaTeX-compiled (to pdf) essays are neatly stored on our home computer, all tagged as necessary for Nepomuk, and mirrored to the remote server. We have a similarly synchronised set of .ical files for calendars and appointments, and vcards for contact information, and of course mail is on our very own setup on our server. As for social networking, an own-hosted modded WordPress install is used for (micro)blogging. We don’t mind a little Facebook here and there, but would also love to be kept connected on IRC. Of course it’s a-given that our remote server(s) are all equipped with (S)FTP, SSH, and Webdav support.

I don’t know much about the current state of smartphones but perhaps for those that do – can I bend it to use what I use and still feel a decent sense of integration? With this I mean drag-and-drop transfers for files and multimedia, seamless switching between local and remote locations (with support for above protocols), directory synchronisation (rsync?), PIM synchronisation as necessary for ical/vcard/mail/rss with a custom and remote location (or at least importing), and perhaps clients available (terminal emulation, anyone?) for SSH, IRC, and streaming.

Can I actually use my services the way I want them on a smartphone in this day and age?

I know I’ve had somewhat limited success on my own aging Windows Mobile phone, with third-party apps accomplishing iCal sync, SSH (putty), IRC, mail and RSS, and luckily it isn’t tied to nonsense like iTunes when I want to transfer music over. However the rest of the long-dead OS shove these few glimmers of freedom away in a dark and dusty corner of the market. I quite honestly wonder how the rest is doing – so I ask again:

Is it possible?


The zen of PIM

korgacPIM is the acronym for Personal Information Management: todo lists, email, rss, calendar, contacts, journals, blogs, etc. Recently I have been poking around trying to achieve the “zen” of PIM, where my PIM data is accessible from anywhere, and from any medium – from the internet, from my Windows Mobile 6 powered phone, and from my desktop.

As a KDE-user, naturally I have attempted to use the Kontact PIM suite. KMail and Akregator both work wonders with my data and are no problem, but working with contacts, calendar and to-do lists are a real PITA. The interface for managing the actual PIM storage (which I’m more interested in than the PIM data itself) is completely unintuitive, making me choose from several backend types with no description whatsoever, to work-in-progress Akonadi migration of which the status is quite unknown to me, to random remote/local synchronisation of untitled .ics files. The journal section seems to serve no purpose whatsoever.

Are there any kind souls who have reached their own personal “zen” of PIM management who care to share their setup with me? The criteria is:

  1. Hopefully able to use with Kontact
  2. Hopefully able to use/synchronise one way or another with my Windows Mobile 6 phone
  3. Not using Google services, but a way to store a compatible-with-other-apps file on my personal server would be a definite plus!

Note that I do not necessarily need a feature packed application. For example for a calendar all I want is the ability to say “this happens on this date”, with an option for start/end time. The repeating event feature is also optional but appreciated.