A scalable and adaptable standardised user file structure?

I’ve got two more mock exams to cover (both on Monday), some University applications progress has been made, more and more of the ThoughtScore project is inching its way into my WIPUP profile (starting to get to the interesting bit), and I’ve opened Blender again to do a quick animation favour which can be seen in my Uncategorised category.

But this post is nothing to do with those fun and amazing things. No. Today I want to talk about something that warrants 3-4 syllable words being used in the post title.

The ins and outs of operating systems aren’t exactly my speciality but I do know that they have a uniform file structure. In Windows, a lot of system stuff goes in C:\WINDOWS, your user files goes in Documents and Settings, and programs go into Program Files. In Windows inside your user file you automatically get a set of folders such as My Documents, My Pictures, etc. In Linux (and all UNIX I believe) your user’s folder is in /home/username/ and other than having hidden dotfiles to store local application settings it’s basically empty. At most your desktop environment or distro will add a "Desktop" folder.

The question is how should I archive and organise my files such that they’re easy to find, easy to transport, and easy to manage. Sure, the Microsoft approach works for some people, but most of the people I know completely ignore that structure. I’ve been thinking about it for a while and have isolated a few ways people normally try to manage it.

By file extension. It’s easy to find and easy to archive. However it’s difficult to manage once you start getting rare file extensions or files with no extensions. It also becomes a pain when your program output or save files require several files of different types to be grouped together. Files can become strewn all around the computer and projects which require hardlinking to files will easily get out of hand.

By file type. This one is only slightly different to file extension in that people group things by what files are, not their extensions. For example "Images" instead of "jpg", "gif", etc. Pros and cons are similar to above.

By file purpose. This is a project based approach. Files are grouped into their uses, such as "Homework", or "Movies", or "Project X". This is rather commonly used but often clashes occur when used in parallel with others such as File extension or file type, especially when the nature of the project requires hardlinking to file locations.

By file attribute. This is often a temporarily used file structure when people are sorting out files. Such examples include "bobs photos", "to be sorted", "jazz music", etc. The main use of this structure is to make it easier to transport the files or manage a bulk collection of files. The sad story is that these directories persist waaaay past their useful lifetime and prove to be effective cloggers.

By time. Useful for archiving, completely regardless of the content. Used for management and for finding, but rarely useful for transporting files.

By referenced location. Some files are put in locations regardless of semantic value and instead for technical convenience – for example dotfiles, plaintext, logs, tmp, backup files, and referenced files from other apps (scripts, programs, graphic projects).

By organised chaos. This is probably how almost every single Windows user’s desktop looks like. A complete mess of random rubbish used to dump stuff. Files are strewn regardless of any attribute and found through searching, indexed searches, and file manager filters.

Now of course searching for the zen of a users’ file structure will only end when a structure is able to accomodate a large number of files, a project-based workflow, technical restraints, archives, and miscellaneous files. In this case it’s useful to define what "accomodate" means, which is an ability to easily find a desired file, to sort and prune undesired or irrelevant files, to prevent duplication, to transport similar files easily (eg: all within one directory), to quickly break down a collection into manageable chunks, and to allow any newcomer to intuitively adapt to your filesystem.

I currently run a structure where my homefolder is where I dump active files, I have a primarily "file type" structure, archives are done through "file purpose" (I despise time), and projects have their own substructures which are completely dependent on referenced locations. For me the biggest inconvenience is referenced locations, where I find myself unable to bulk manage files simply because of the inconvenience of have to re-reference their location. The rest is chaos. All in all, miles away from my personal zen.

Anybody who’s achieved a personal zen are welcome to share.


Make a category not considered as a post in WordPress

In other words, how do you make posts that are in a certain category not count towards total page post count in WordPress?

A while back I set up Asides on this blog. The problem was that previously I was displaying 5 posts per page. Now with asides it still displayed 5 posts per page, but as asides are probably one sentence long at most I personally don’t consider them to be blog posts. This meant that it didn’t display 5 “real” posts per page. So, how do I fix this?

Disclaimer: I’m not experienced in the under-the-hood of WordPress and as a result some of this solution might be hackery. However it works for me, and that’s what counts.

Problem 1: displaying 5 real posts per page regardless of how many asides there are.

WordPress loops through a series of posts per page and displays them one by one. Initially I thought they would increment a counter, of which I could easily change so that if the post category was in “asides”, it will not increment the counter. However there were a couple flaws: 1) There was no counter, or I completely missed one, 2) The database queries sets the LIMITs from the administrator settings right at the very beginning, and 3) pagination will be completely messed up.

The solution was pretty simple, firstly we set the database query LIMIT to an obscenely large amount – more posts than we ever think we’ll need on a page. This can be done in the administrator panel. Change “display posts per page” to a random large value. I chose 15 because it seems pretty realistic that real posts + aside posts < 15 for 99.9% of the time.

The second step is to manually change the criteria for when the loop terminates. This way it will not actually show 15 posts, but instead up to 15 posts. What we’ll do is create a new counter, where ever time we display a post that isn’t an “aside”, we increment the counter, until it hits 5 posts (if I wanted 5 real posts per page) – at which time it’ll terminate the loop.

This can be done in the wp-includes/query.php page. To begin with we’ll need a new variable in the class for our counter. So below class WP_Query { we should add:

var $counting_up = 0;

Just to make sure that $counting_up resets itself as it should, we’ll add this to the init() function:

$this->counting_up = 0;

Now the next step is to modify the the_post() function. When the loop has started and the category is not an aside, we’ll increment our counter. In this example my aside category ID is category 429. This will be different for you, so you change it. So simply add this to the the_post() function:

if ( !in_category(429) && $this->current_post != -1 ) {

Now we’ve got our counter, we’ll set up the loop to terminate correctly. This can be done in the have_posts() function. Notice this is the have_posts() function inside the WP_Query class, not outside. We can modify our if statement to terminate when our counter hits 4 posts (as the first isn’t counted – therefore in effect we’ll display 5 real posts), and also when we don’t have a do_not_terminate variable set to the WP_Query. Why this do_not_terminate variable is important is if we ever need to override this, as well as later I’ll explain when we look at pagination issues. Here is my completed modified if statement:

if ($this->counting_up == 4 && !$this->query_vars['do_not_terminate']) {
$this->in_the_loop = false;
do_action_ref_array('loop_end', array(&$this));
return false;
} elseif ($this->current_post + 1 < $this->post_count) {
return true;
} elseif ($this->current_post + 1 == $this->post_count && $this->post_count > 0) {
do_action_ref_array('loop_end', array(&$this));

Now we’ve solved problem 1, and 5 real posts are displaying on our front page, let’s move on to problem 2.

Problem 2: previous page, or going to older posts will no longer work.

Since pages are pretty obsolete at this point, we’ll switch to using offsets. This is because each page will no longer display a fixed number of posts, each may display a variable amount of posts, minimum being 5 (that we set just now), and maximum being 15 (that we set at the very beginning). So to start we’ll hop over to our index.php in our theme file, and simply get the offset from the URL and pass it through to our post loop. Here goes:

<?php if ($_GET['offset'] && is_numeric($_GET['offset'])) {
query_posts('offset='. $_GET['offset']); $offsetting = $_GET['offset'];
} else { $offsetting = 0; } ?>

So with that code in index.php, we can now visit myblogsite.com/?offset=20 and offset our posts by 20. To determine how many posts to offset by in previous pages, we simply take how much we’re currently offset by, and add all the posts we’ve displayed on the page, regardless of whether or not it is an aside or a real post. To do this we need another counter. So we’ll initialise our counter, perhaps near the beginning of index.php:

<?php $on_page = 0; ?>

… then within our while (have_posts()) { loop, (or whatever equivalent loop your theme uses), we’ll just increment it:

<?php $on_page++; ?>

So then we recode our “previous posts” link to go to:

<a href="http://yourblog.com/?offset=<?php echo $offsetting + $on_page; ?>" >Previous posts</a>

That was simple, eh? This brings us to problem 3.

Problem 3: newer posts don’t work, for obvious reasons.

Going forward in time is a little bit more complex. We want to calculate how much less we should offset by. To do this we’ll create a function to calculate this. The function will need to know how much we’re currently offset by. Based on that, it’ll query 15 posts into the future, then loop through those posts in reverse order. If it can’t go 15 posts into the future (eg: on the first page, and perhaps the second), it’ll go as far into the future as it can. When looping through, it’ll record the category of each of the posts. Whenever it hits a post, it’ll increment the count we want to offset less by. When we hit a post that category isn’t an aside (category 429 in my example), it’ll increment a counter that determines how many real posts we’ve hit so far. So therefore we have two counters. When the real counter hits 6 posts, it’ll terminate the offset counter. This is because I want 5 real posts per page, and based on how we coded problem 1, we know that the last post of any page must be a real post, not an aside.

We can place this function in the functions.php file of our theme. Here is the function, of which lazy people can copy and paste:

function back_to_the_future($offset = 0) {
$new_offset = $offset-15;
if ($new_offset < 0) {
$new_offset = 0;
$diff_offset = $offset - $new_offset;
$future_query = new WP_Query(array(
'showposts' => $diff_offset,
'order' => 'DESC',
'offset' => $new_offset,
'do_not_terminate' => TRUE

$post_data = array();

while ($future_query->have_posts()) {
if ($diff_offset == 0) {
} else {
$cat_id = get_the_category();
$cat_id = $cat_id[0]->cat_ID;
$post_data[] = $cat_id;

$post_data = array_reverse($post_data);
$count_posts = 0;
$count_total = 0;

foreach ($post_data as $post_cat) {
if ($post_cat != 429) {
if ($count_posts == 6) {
} else {

return $offset - $count_total;

People who looked through the code will realise that we passed the do_not_terminate variable to WP_Query that we set up when addressing Problem 1. This is required because if we didn’t, we won’t get 15 posts into the future, instead we’ll just get however many posts starting from 15 posts into the future that include 5 real posts – which is totally useless.

To finish off nicely we’ll edit our “newer posts” link to use this calculated offset in our index.php file, but only display when we have a proper offset to show and we’re not on the first page.

<?php if ($future_offset != 0 || !empty($offsetting)) { ?>
<a href="http://yourblog.com/?offset=<?php echo $future_offset; ?>">Newer Posts</a>
<?php } ?>

Tada – all done! I hope that helped somebody out there, but if not at least I have it for archival purposes. If you’re using it, let me know how it goes!