letting things come to me

Since I've become more aware of the Web2.0 revolution, I've noticed more and more web technologies that I'd been thinking about years ago rising to the surface. A key example was an idea I had about online libraries where you list your books, and then other people can browse your library and if appropriate make requests to exchange or borrow books. It would be like an enormous communal, virtual bookshelf, except you'd get to read real books instead of reading off a computer screen (which despite what anyone says is still less comfortable than reading a book! at least to a majority of users...).
Well, now we have Listal, LibraryThing and AllConsuming. I'm not sure who owns them and whether I really want to commit my data to them, but the services are definitely there and doing what I thought such services should do years ago.
Which brings me to my main point. With services like these, I shouldn't have to think about committing my data. My data should reside where I want it to, and I should allow these services access to my data on my terms and conditions.
We need some sort of a platform for maintaining information, and then transforming it and submitting it. Preferably in a relatively extensible and/or standardized way. XML and XSLT style technologies seem to be screaming out to be used in this sort of position.
In addition, we need some sort of voluntary code of conduct whereby we can be reasonably assured that we can reliably dictate the terms under which such services can use our data. Maybe some open source datakeeper software modelled on recent digital rights management advances, so that as well as records companies being able to control our rights on the music we license from them, we can also revoke other organisation's rights on the data they license from us.
Digital rights management isn't necessarily a bad thing, but biased towards the goals of the powerful it is clearly not a good thing.
So the data landscape of the future? Data residing in multiple incarnations on various storage devices across the world, controlled by open source datakeeper software allowing only authorised people to access it, and transform it, using flexible tools. The owners of the data - you and me in addition to the organisations and corporations - empowered by our data's newfound mobility and flexibility.

some musings on data and interpretation

Some ideas that came out easily, on the 9th of June 2005. It feels like they're going somewhere, but not without some thought, and probably a lot of maths and programming.

• Data is inextricably linked to the methods that process it.

• Memories have a language of their own, do humans share the language of memories? Could a goal of humanity be the effective translation of our language of memories? Could imperfection in translation be a huge source of conflict too?

• Information is symbolic, without a means of interpreting the symbols the information conveys nothing.

• Once the symbols can be distinguished, patterns within the symbols can be accessed. However, the original meaning (intent) may be lost and will generally be distorted.

• Given a set of symbols and an interpretative mechanism for those symbols, to what extent are the patterns spotted a result of information within the interpretative mechanism, and to what extent are the patterns spotted a function of the information within the symbols? I rather suspect that this question also misses a point: It falls foul of the fallacies of subject/object metaphysics. Within any interaction there is participation from both sides. For an interpretative mechanism to detect patterns within symbols there will have to be a contribution of information from both sides, at some level.

• Interpretative mechanisms range in style: some seek to minimize their input of information while maximizing the effect of the external data, others use external data as a randomizing element or mixing agent for expression of their own internal data (maybe this is a good framework for interpreting the occult/astrology/science etc.?).

• An interpretative mechanism which includes the assumption that it does't affect the data it processes is ultimately flawed. (See subject/object metaphysics). Science often falls foul of this.

• Data with no obviously associated interpretative mechanism is worth less, all other things being equal, than data with an interpretative mechanism.

• High quality data may be restricted by a low quality interpretative mechanism, and vice versa etc.

• Data can be represented by a set of symbols. An interpretative mechanism can be represented by a processor for that set of symbols, which may generate another set of symbols in response, or may create some physical output or whatever.

• Data never exists as a set of symbols (except in the universe of platonic forms...). Instances of data almost always have some elements of the interpretative mechanism held locally. That is to say, the interpretative mechanism has a large influence on the manifestation of the data. Usually data and an interpretative mechanism co-evolve together, and are intimately interconnected, even if only at substrate levels (eg, dependence on ASCII, or spoken language or whatever).

• An interesting model of reality is merely a seething array of interdependent information. The extent of this information is phenomenal, and the levels of structure range across the orders of magnitude widely.

should we be suspicious of google?

"Google's mission is to organize the world's information and make it universally accessible and useful." - Google's mission statement

I would be a lot more comfortable if it was Google's mission to assist in creating the technologies that will free the world's information. As it is Google has simply asserted that it wishes to become the monopolistic broker for what is fast becoming planet earth's most valuable resource. Attached to the information Google wants to organise and serve to us is, potentially, almost all the value (of any kind) tied up on the planet. This isn't so hard to believe when you see the proliferation of gadgets, and monitoring gadgets, everywhere, and their increasing connectivity to open networks.

I'm an optimist, and I don't think Google can survive much longer behaving in the way it does. Just as I believe Microsoft is a dying monster, serving poorly crafted computing products to the illiterate computing masses out there, I believe Google will eventually go the same way, except I think Google will probably have left a much more valuable legacy in terms of experience, lessons learnt and contribution to the internet and activities undertaken thereon.

What we need is decentralized, open-source search. The days of concentrating this kind of resource in the hands of a single company really has to be over. If we stop kidding ourselves this is clearly the most sensible, and the only possible, way forward. One Google is a single point of failure for one of the most important resources on the internet.

Distributed search composed of hundreds of flavours of search engine scattered all over the planet, even on people's home computers and househouse appliances etc. would be a much stronger system, and would in my opinion have the potential to be a lot more trustworthy than our current centralized solutions.

Check out lucene/nutch. I will soon, it seems to have developed a little in the last few months.

Maybe we'll all be saved.

Google Maps and GMail and all the various Google-Wows are surely amazing, but not as scary as what they all inevitably point to.

So where does that leave all the big internet portals?

In my mind it has to be something along the lines of: decentralize-and-open-your-source or bust. There's no reason why one company can't produce a bulk of the technology and even benefit in a big way financially from it, but I think in this day and age it mustn't do it behind closed doors.

Security and trust demand it, and morality is begging for it.

Roll on the next few years!

the manchurian candidate

I just finished watching The Manchurian Candidate. It's quite an interesting film. And it was quite interesting to watch it. I was watching it on a video-on-demand service, I could pause, rewind and fastforward the movie. Everything was streaming into my living room through co-ax. My TV is now connected to Media Player running on a dedicated box, attached to the internet. Basically.

It doesn't seem so long ago that these sorts of technologies were pipedreams.

Now they're everyday, the preserve of the geographically particular, and they come complete with laggy update and slow loading. What happened to timed IP? It's coming I guess.

The film, that was interesting too. A fairly simple recombination of old themes and current media foci, proficiently brought together in a very easy to digest whole. The visuals were nice, the mood was just right, and the visual cues were perfect, there was no clubbing you over the head with the key plot points, but everything was present and correct. As far as I could tell.

It got me thinking. [minor spoiler ahead] The part where Shaw kills the senator and his daughter seems so out of place at the time, but then the revelation that his ambitious mother put him up to it [end of spoiler] serves as a timely reminder that human nature hasn't progressed in the same way that technology has. Visions of cavemen in shirts and ties, turning up to work with swish laptops and fancy watches spring to mind. Yay for caricatures. Then we watch those cavemen running the United States of America, and telling us what it means to be moral. EDIT: Not forgetting that in this instance 'caveman' refers to the common consciousness of caveman, not the less politically incorrect view of 'cavemen' that I prefer. Why oh why did I succumb to such prejudiced concept choice? Ho hum...

The film also seemed to be a veritable montage of idealistic political opinion and suggestion. It seemed to me that virtually all the background material had been carefully selected, including the news tickers on the various TV broadcasts and the contents of background TV broadcasts. Having one of the key characters walking through a school play fitted in well with the context, and the parallelism was very apt. The brainwashing in the film was clearly exaggeration for the sake of making a point, but the parallel with school kids provided a neat stepping stone onto some more down-to-earth problems.

The film clearly seemed to be a platform for peddling left-wing viewpoints, wrapped up in current issues and parlance. It makes me wonder to what extent 'meme-placement' in films is becoming commonplace. And to what extent it is moral or not.

An absolutely neutral film would be verging on impossible to produce, and would probably taste like the plastic food we're all getting accustomed to, so why not introduce all sorts of background elements that fit in with your message? I think the key point is that subtly choosing background elements smacks a lot of attempting to subliminally influence the audience. But even if you don't subtly choose anything choices are made, by people, and choices are never neutral, maybe especially when they aren't consciously considered. So which is better, being prey to the memes that propogate themselves under the radar or being aware of what you're transmitting and accepting it?

Nowadays I'm inclined to go down the road of awareness and acceptance, rather than ignorance and denial. (Although I admit the discussion is a lot more complicated than that, and that I've loaded the description badly... ho hum. Just my honest-memes going and making my writing clumsy and 'transparent') And I'm also inclined to say that the argument that proposes that this is unfair on people with insufficient capacity or attention to vet their senses is a dodgy argument. People adapt to their environments, and as long as we have unscrupulous people propogating dodgy self-serving content it's up to those with a conscience to outrace them as hard as possible. And we have a big advantage: we can work together much more, and much more transparently.

a brief manifesto (for myself)

The computing substrate seems to be improving at an ever increasing rate. But there are some things that I think need improving, some on the level of user education, others technological. Here are a few that spring to mind:

• Provision of parallel architectures that are actually properly designed
  We have amazing bleeding edge facilities nowadays. Progress is rampant, and in all directions, like bacterial colonies on agar jelly. We have various standards groups like the W3C trailing behind doing their best to mop up the spillage, but how good is it? The ideas mill is in overdrive, now all we need is groups to set these ideas in their proper places.
  I'm not advocating design fascism, but I am advocating the more widespread rolling out of bulletproof architectures that the military wouldn't be afraid to use.
  The secret to success is to realise that there are only a very few tricks in the book, but that those tricks are extremely powerful. The ideas mills churn out specific permuations. Someone needs to run along behind spotting the underlying patterns, and making simple but highly generalised tools that the rest of us can use without wading through piles of barely distinguishable competing 'standards' and hacks.
  Any contenders that I'm not aware of yet?
  

• Data awareness
  Our data shadows are burgeoning. Technologies need to be developed that protect our data, and maximise its effectiveness. I have separate user profiles on multiple social websites. When will I be able to store my profile locally, and allow websites to access it when I wish, and allow them to store their own copies only if I allow them to? Decent data management would make the computing substrate so much more useful. I wouldn't have old data all over the internet crying out to be maintained. I could participate in a far wider variety of stuff. Service providers could concentrate on what they were good at, not on recollecting all the various data that everyone is tired of giving for the umpteenth time anyway.

• Better documentation
  All due respect to the wikis of this world. But the linking systems are beginning to show their age. We need real databases with real tagging systems, and proper diagram support. Why are we still relying on bitmaps on the web? Macromedia's solution is a poor stopgap. What happened to SVG? A few more diagrams would help reduce the mess the ideas machine spews out, and improve quality for us all.

In a word, integration. We need to allow everything to be a whole lot more cohesive and consistent, while maintaining the exciting churning hotchpotch of creativity that provides us with the ideas that need organising in the first place.

making use of the web

There seems to have been an explosion of free internet services, the ones I'm aware of are taking advantage of Flickr's apparently open database policy (a trend I'd like to see, and expect to see, repeated across many of the world's big, and upcoming, social websites), Google's various projects (maps, gmail*, blogger, adwords/sense etc. etc.), and popularity in interface/database concepts such as folksonomy/tagging (del.icio.us), wikis (wikipedia) and other technologies designed to enrich the cooperative aspect of the web.
Now seems like too good an opportunity to miss, maybe akin to the .com era of yesteryear, but this time I feel like it's going to be at least a little more sustainable (especially so, given that I'm well aware that I'm no early adopter here!). The internet is turning into an enormous, and multi-faceted, cooperative resource. Taking part in that seems like a very attractive option.
The choice, however, is bewildering.
Should be an interesting ride!
*feel free to contact me for a gmail invite :)

loose thoughts on memory

You could think of perception as pattern matching. Matching patterns of mental states against other patterns of mental states. For example, perceiving a banana could be viewed as matching the 'live' perception of the banana on your senses against the stored perceptions in your memory. Without your memories you wouldn't, from your own perspective, be perceiving a banana, you would be perceiving something unknown, and you would be formulating a new memory for which you could later fill in the name 'banana'.

(nb: parallelism in pattern matching 'schemes', and fuzziness of pattern matching. Explore methods of pattern matching.)

It is quite easy to divide perceptions into live perceptions and stored perceptions. Live perceptions are perceptions that are currently being perceived, that are immediate, arriving through the senses directly. Stored perceptions are those that have been perceived and remembered, and are accessible for remembering again.

(nb: continuum between 'live' and 'stored', also stored perceptions can be viewed as budding off new live perceptions every time they are accessed/perceived again, leading to an expanding, evolving tree of memory.)

If perception is viewed as pattern matching, as matching patterns of mental states against other patterns of mental states, then combining that with the division of perceptions outlined in the previous paragraph, 3 obvious combinations become apparent.
We can pattern match between immediate perceptions, a sort of 'on-the-go' perception, we can tell that one animal in view is of the same kind as another animal in view, or that there are two similar forks side by side on the table.
We can pattern match between immediate and stored perceptions, ie, the fork on the table looks like a remembered fork.
And finally, we can pattern match entirely with the realm of stored perceptions, we can imagine various items of cutlery entirely with our mind's eye.
We can imagine complicated dancing patterns of this kind of pattern matching going on in our every day perceptions, and analysing these dances can shed a lot of light on the way our consciousness could work, as well as providing a neat framework against which consciousness could arise out of basic sensory mechanisms.