The State of GLScube

It has been sometime that there was any public activity at the GLScube front. The lack of updates were a combination of how some of the developers became too busy, and some other points that I would get to next. But first, let me tell you GLScube is far from being anything but alive.

When we have started development on GLScube, we had the one goal of finding a better user experience in organizing and retrieving documents. That simple goal was then broken down into many technical challenges, and we went on to solve each one of them. As the project progressed, however, we came to realize that we were focusing too much on the technicalities, and always favouring technical decisions over usability-related decisions. Here, I will introduce you to those two issues, and why we reached a point were we have almost “stuck”!

Let’s first get to the usability-related issues, since after all, the project was made in the first place to enhance on usability. One of the most important features in GLScube is Virtual Collections, which are analogous to the traditional directories but whose contents are retrieved at runtime based on an associated query. Some Desktop Search engines, like Apple’s Spotlight in Mac OSX, have a similar feature implemented, though only presented to the user as an extra feature he can use in addition to the traditional directories!

This to me is certainly an issue. The concept of convincing users of using both static and dynamic means to organize their documents is utterly ridiculous. It is already hard enough to teach an average person about the concept of hierarchical directories and how he can walk through them, and it definitely would be very difficult to explain to him what a search query is and how dynamic directories work, so it is bewildering to imagine we would have to explain to him both. The problem is not just in explaining, it is in convincing. It is easy to put features and tell someone about them, but to (politely) force them into using them is something else. So, when designing the concept of Virtual Collections in GLScube, our ultimate goal was to wipe out traditional directories altogether, and in no way wanted the user to use both.

This is easy to do, and easier to say. But then what about the migration phase! Let us say that in 10 years time, everyone in the planet would be using GLScube, then till that day comes, we would have to maintain compatibility and interoperability between systems that use GLScube, and systems that don’t.

To make this more clear, let us say you created a Virtual Collection called “music” that, obviously, listed all of your music files. A friend came and gave you a CD containing some new song, you copied the song and put it in your GLScube Store. We ultimately don’t want you to specify any file path, and the song will, as you expect, always appear in the “music” Virtual Collection”. So where is the problem? The problem is in the answer to where that file was stored by GLScube. Since GLScube augments the traditional file system, it has to store it on the file system as a traditional file, which is really not a big deal, except that we would have to store it in an adhoc location. In other words, if the user wanted to retrieve this file from the underlying file system, say if he moved this drive to a non-GLScube system, then he would have a very hard time trying to find this file, which may even be named in a different name (or post-fixed with some value to distinguish duplicate names, or worse, placed in a some node in a hierarchy of directories, to avoid storage of too many files in one directory).

There are many such questions that have several answers. Like whether a user should be able to create a hierarchy of Virtual Collections, or that would put us where I started? Or, when a user decompresses a file, how would we deal with its parts that may depend on their hierarchy and each other?

This uncertainty in targets leads to uncertainty in design decisions, and also affects the scalability of the code.

Code. Code is another issue altogether. With the most critical point being consistency. Since GLScube runs in the user-space, and relies on inotify for receiving notifications on changes in the watched file systems, this kind of decoupling makes GLScube very susceptible to inconsistency due to unhandled events due to reasons like, most seriously, overflowing of the inotify buffer, and hence never knowing that some events even occurred!! I used PostgreSQL, which turned out not to be the best decision. Add to this some large number of stability issues, and it is suffice to say that the code base is not as good as it could be.

Now. Now we have two directions we can move to from here. We can continue working on the 0.1 prototype, or, we can start over! Starting over is not just about code-rewrite (which the current code base would really benefit from), but also a re-evaluation of all design decisions, mainly from the usability point of view.

I am most inclined to the second approach. The 0.1 prototype provided a very clear picture of what issues are in this type of project, and the priority of the challenges in it.

To learn from our own misfortune, it is clear that the essence of the problem is how to solve usability correctly. And since a few should never be allowed to decide on how others should use a software, I am thinking about an open-way to do a usability study; some rapid-prototyping method to put rough mockups for all the possible features and receive feedback from the open source community on what works better.

Note: For people who want instructions to compile the 0.1 prototype, we are preparing those and will put them together as soon as we can!

2 comments October 16th, 2006