Month: April 2013

Curation and the inevitable data apocalypse

I have a lot of data. Not, I am sure, a lot by the standards of an experimental scientist or someone who shoots a lot of video or takes a great many photographs, but a lot of small files which I don’t want to lose.

I have also had a number of data losses over the years – some from media failure, some from user error*. The most catastrophic was a motherboard failure on an old PC In the days when storage was expensive and removable discs were tiny. The computer worked for the most part, but there was a fault which meant that it could not calculate checksums correctly. The first things to go were the compressed directories that I had set up so as to make better use of the hard disc – that was the signal to me that I needed to do a comprehensive backup. Which I did, very carefully zipping up all the stuff I cared about most onto multi-floppy archives. An operating system reinstall did not fix the problem, and I was more than a little put out when the computer came back from the mender and I found that every single one of those archives was corrupt, the cherished data lost.

I suppose I should be glad that this happened before I had a digital camera, but I lost a lot of writing.

I have just performed an upgrade on my laptop to allow more current software to run on it, and was looking at my hard disc to clear things out a bit. What I found was nested machine images, copies of previous computer hard drives resting inside each other, copies made as I have moved from machine to machine to machine: yig contains wendigo, which contains ithaqua, which contains dagon, nyogtha and daoloth – generations of computers going back fifteen years. There’s even a disc image from the first machine I owned with a hard disc in it, kurt the Acorn Archimedes, although that was recovered in a recent bout of data archaeology.

Each previous image contains the seed of the data I have on the system that superseded it.

I’ve been using the same basic layout of my data directories for a long time, so I can trace the development of projects over the lifetime of these machines. I have early versions of programs and websites that I’ve maintained over this whole period, and larval versions of stories, and early snapshots of photo directories. I also have multiple copies of large data projects, like the effort to digitise my vinyl, which at least means I haven’t lost anything.

The question really comes down to how much of this ancient data I want to keep around. The large data projects I want to make sure are backed up in a couple of places, but the prehistoric data directories? Not so much, I think – make sure nothing is unique, but I feel less need to retain these old versions than I once did, when I started a repository project which was intended to act as a versioned history of all the work I have done.

Half the data on my hard disc on yig is duplicated, or stale, or just doesn’t need to be available at all times.

Time to clean up.

[*] aka PEBKAC – Problem Exists Between Keyboard And Chair

One Response

Lurgy Town

No post today – I had a cold land on me late on Wednesday, and it is wreaking its little viral carnage upon my system.

Is it just me, or colds get worse as you get older?

In the mean time, let us continue that theme: what plague would you visit upon a world or a country or a town to challenge your MC? Is it curable? Is it even fatal?

Leave a Comment

The Right Tools

Some people obsess over books about writing, some distract themselves with tips lists, or blogs, or irrelevant media. I distract myself from writing by worrying about tools.

I like tools. I have spent a large chunk of my programming career writing tools, whether for me or others to use. And there have been a couple of times in my writing where I have focussed on the tool that I don’t have rather than the writing that I’m not doing.

The first time this happened was when I had an idea for a story called Ships of the Desert. The premise was that of a world where the Mediterranean Sea had not formed: there was a vast desert rather than a sea – the mightiest of deserts, what the Sahara would be with no pesky water in the way. The consequence was that the coastal cultures around the Mediterranean never formed, and the Euro-Afro-Asian land mass was dominated by Viking peoples in the North, and Afro-Asian peoples to the South and East. The eponymous ships were low-friction skiffs used to navigate the trackless wastes of the desert.

This was a promising enough idea, and I had a lot of fun doing research on Scandinavian and Native American* cultures, but I ran into the problem of not knowing how to organise my notes. I hit upon the idea of a database cum hypertext editing tool. I even started writing this tool (called DataFrame), but this was in 1990 or so with no Internet available to me so I never got very far. In fact, I’ve never quite found anything that does what I wanted then, although wikis come close. The upshot was that I ended up with half a text editor and some scraps of novel ideas scattered across a few text files.

Hankering for the tool killed the writing.

I am struggling with a similar hankering now. I want to draw diagrams that I can use to describe relationships between characters, events, locations, and other story components. I want diagrams which expose the relationships but allow me to add metadata attached to the diagram elements: what I want is a UML-like diagramming tool, but for story rather than software.

Now, there is an open source tool called Dia which supports creating custom shapes that can support exactly this kind of metadata. It doesn’t run on my Mac since its version of OS X is too elderly, but that’s just a matter of an upgrade and then I could define the shapes and draw the diagrams and…

But no. Not now.

What I am going to do is write this in a spreadsheet or a database app of some kind, probably just a spreadsheet in the first instance, just to get the ideas down.

Because the story is more important than the tool.

Addendum: Scapple (from the Scrivener folks) looks quite promising for the kind of loose diagramming I want, but not sure it supports the hidden notes part. Worth a try at some point, but again it needs OS X 10.6 to run.

[*] don’t ask.

One Response


Every story needs tension, and one mechanism for supplying that tension is a conflict with an antagonist.

Conflict here doesn’t necessarily mean a physical confrontation, but rather another character whose goals are not compatible with those of the protagonist.

The etymology of the words is interesting. A casual reading suggests the terms come from “pro” and “anti”, that is for and against, but in fact protagonist is from Greek words meaning “first actor”. Antagonist comes from “anti” as one would expect, but in dramatic terms an antagonist is defined by their opposition to the protagonist.

The nature of that opposition can change over the course of a story, Let’s say the MC likes to collect little frog figures. The antagonist might be another collector who wants the same rare figure, the one with the golden stripes and the foil-backed jewel eyes.

This antagonism could lead in all sorts of directions. The antagonist might want the frog as a way to attack the protagonist. Or the antagonist might want all the frogs and the antagonist is just coincidentally looking for the same thing. Or the antagonist might just collect frogs as a side business, and really they’re an evil mastermind who wants to take over the world!!!

(somehow, that last statement required three exclamation marks)

Introducing an antagonist whose goals are larger than the protagonist’s initial interest is a good way to grow a hero. Let’s say that our protagonist learns of the antagonist’s world-domination plans after losing the rare frog, and that the antagonist must be thwarted. At some point the antagonist should learn of the earlier interest in frog figures, and offer the protagonist the rare frog as a way of buying them off.

Earlier in the story this gesture might have worked, but now the protagonist has grown and sees that stopping the antagonist is more important than obtaining the toy frog.

This is obviously a silly example, but this idea of the protagonist uncovering deeper and darker truths about the conflict he or she is part of is quite common. If nothing else, a story where the protagonist knows everything is not necessarily going to be an interesting story.

An antagonist doesn’t need to be a character in the story per se: the protagonist might have the simple goal of getting home, but the weather can be the antagonist here, constantly upending the protagonist’s plans.

For my own writing, I have been working on making my antagonists more well-rounded, considering them to have their own story arcs which my writing offers small slices of. So in Bluehammer, for example, the priests who are seeking power have back-stories just as the protagonists do, and just as other recurring characters do.

Which of course is what I am trying to do with Song at the moment, too, and which I should get back to now.

One Response

Rereading the winner

I spoke too soon about having a winner. Newsblur has been… blurry. It was showing me articles as new that had been posted two days before, which was damaging my confidence that it was a stable tool.

Also, the Android app is huge.

So, I’ve switched over the Feedly which has a much cleaner look and a more responsive UI. The Andoid app is about a fifth of the size of the Newsblur one, and can be moved to the SD card (which seems to be an increasingly unusual feature, but I miss it hugely when it’s not implemented). My hesitation before was in needing to install a browser plugin on my computer, but that was a lot less intrusive than I feared. And it does authentication through Google so no need to create a separate account (and of course extracting Google Reader state could not have been easier).

So far, so froody – it has a refresh button which makes me feel better, and I don’t feel like it’s lied to me yet about articles being available.

Any other apps that you’ve found useful to replace the lamented Reader?

Update: another win for Feedly over Google Reader even is that the app is available in the Amazon app store so I can put it on my stock Kindle. Quite pleasing.

Leave a Comment

Rearranging My Storytelling Approach

The way I have always written roleplaying games has been to describe the setting and the supporting cast (NPCs – non-player characters) with more or less detail. Depending on what part the NPC is going to play, it may be described with a stat block – a collection of numbers describing the skills of the character in terms of the game system. Then I let the players loose. There are certain events that their actions may trigger – for example, open the safe and the alarm goes off, bringing the police down upon them – but the basic structure is that certain items or pieces of information are found in a particular place or with a particular person.

I’m going to try changing that.

This is influenced by hearing about Trail of Cthulhu, a roleplaying system intended to focus on the investigation portion of the Mythos gaming experience. The core element is that there are clues, and the player characters will get those clues at some point based on their actions, but the precise location or person is not set in stone. Another important element is to say that the next clue is only available after certain conditions have been set up – and some of those conditions may be the finding of other clues.

In software terms, I would call this a gating condition. I use this in state transition diagrams in particular, where you are defining the behaviour of a system in terms of what has already happened: what has gone before effectively defines what can happen next.

I am also trying to apply this to my novel. The current work I am doing is to take the scenes I’ve already written in the outline and to describe the transitions and changes described by each scene. I also want to describe the major events in the same way. The point of all this is that every scene must have an effect on the story, or the characters, or the circumstances. If it doesn’t have an effect, why is it there?

The early casualties of this approach are introduction and bridging scenes – a scene where the MC wakes up and takes a phone call is not urgent enough to hold the attention; a scene where the MC reviews video footage and checks his equipment is not building tension.

There’s an element of pacing here, of course: every scene shouldn’t be an action scene or the reader will become fatigued. But breaking things up with slower scenes later is a lot easier when the story is good already; transitions and changes can be moved around too.

Have I made new words recently? No – but I do feel like I am learning more about analysing story and narrative.

One Response

And the winner is…

It still rankles that Google Reader is going away, but I seem to have found a viable replacement.


I’ll be candid here: I don’t like it as much as Google Reader. Google’s product has the profound advantages of simplicity and responsiveness – Newsblur does not update as readily as I would like, and I am not all that comfortable with the UI conventions.

However, unlike the other candidates I’ve looked at, the site works in a browser without special plugins, and there is a functional app to run on my phone.

The biggest problem with Newsblur is the feed limit imposed on the free account: you can only have 64 feeds. But honestly if this service continues to function I will probably stump up the annual subscription cost of $25 or whatever it is. With any luck the service will stay in business.

Leave a Comment

In bits on the floor

I’m back to outlining, and I am still trying to figure out a process that works for me.

I’ve been trying a few things:

  1. make the MC more active. I have a strong tendency to be an observer, and my central characters often start out reactive rather than active. Finding appropriate goals for the MC has helped.
  2. define the antagonist. The structure I have for the story is better than it was, but the antagonist was incomplete. Filling in the gaps has given me some good ideas.
  3. introduce conflict. I have a strong tendency to be complaisant, to rub along with others and to just want everyone to agree. That makes for dull stories, though.
  4. the hero’s journey. I’m not a great fan of the hero’s journey as a story structure (I find the concept of a hero in and of itself troubling), but it does embody one important element which helps all stories: the protagonist is changed.

All of these are helping me to craft a more involving story. I made a breakthrough on the ending yesterday, figuring out how to give the MC agency rather than relying on others to act, and that’s pretty exciting. Since I have a better start to the story as well, all I have to do now is work from either end until they meet in the middle.

Do you have particular outlining and story development strategies you use?

Leave a Comment

But Why?

“It is the question that drives us, Neo.”

– Morpheus, The Matrix

I have young children. They ask a lot of questions.

The archetypical question from a small child is “Why?” My kids are a bit older and therefore a bit more sophisticated in their questioning (one recent example – “What is algebra for?”), but their questions remind me as a writer that I need to keep asking questions of my work.

Why does the antagonist attack that vehicle when it’s a high profile target? Why does the protagonist choose to spend time online rather than with his wife?

There are several kinds of questions here, such as:

  • what aspect of the character leads them to make a particular decision?
  • what element of the setting makes this a reasonable decision?
  • if it’s not a reasonable decision, what pressure drives the unreasonableness?
  • what purpose does this event serve in the story?

… and so on.

I’m still working on the outline for Song, and I am asking a lot of questions like these, and others that only make sense in the context of the setting. But the answers are illuminating, and that is what makes the questions worth asking.

Do you have questions which you ask yourself? Or which your work triggers in your mind?

Leave a Comment

Changing voice

I’ve been struggling to write lately. Part of this has been a time thing: projects and vacations have meant that I haven’t been getting up early enough in the morning to write, and that hurts the amount of time I can put in. However I have also been having difficulty immersing myself in the story I am trying to build in Song.

Outlining is hard.

That’s not all, though. I’ve also been trying to avoid a complete rewrite. In building up for camp, one of the steps was “plug existing text into new outline”. When I tried to do that, though, the text did not match up well enough. The opening was wrong because I have changed the location of the MC’s business; the events in the later portions of the book didn’t mesh well with the scene progression from the first draft; and so on. I gave up, and that was part of why I gave up on Camp too (although not having a clear enough outline was the larger part).

I realized the other night that I may just have the point of view wrong. Song is written with a third person voice, but the POV is close in – the narrative rarely leaves the side of the MC, and we get insights into his head all the time. Maybe, I thought as I dropped off to sleep, the POV should be first person instead.

Changing POV is something I have tried to do before with other stories and failed, but this feels right: I can use the MC’s own voice to explain his visual intrusions, his concerns and fears, so that the reader will be able to follow along better with his adventures.

There’s no avoiding a rewrite now!

I still need to get the outline done, but this change of voice feels like it is going to help a lot.

Leave a Comment