Damian Brady's Blog

My last post talked about how I wasn’t getting anywhere with my project.  Well now I am.

I might have mentioned that I’m using php and while it’s pretty easy to do stuff, I sometimes feel uncomfortable with the looseness of it all.  If you’ve used php in addition to a “real” OO language, you’ll know what I mean.  Lots and lots of globally-accessible functions with no namespaces, duck-typing, all that sort of stuff.

So I’m following some better practices (as suggested by Jon in his comment on the last post).  Nothing is very groundbreaking and I’m sure most newer php projects use the same kind of things.  Anyway, here’s what I’m doing:

I’m using an ORM library to get from my database to my code and vice versa.  Specifically Doctrine.  I’m reasonably familiar how these things work having used NHibernate for .Net (a port of Java’s Hibernate), but I’ve been impressed with how easy it’s been this time.  I got Doctrine working beautifully for my spider-web of a MySql database inside an evening with no experience.  I’ll let you know how it works out in the long run.  And yes, MySql is the database right now but Doctrine allows it to change very easily as expected.

I’m using an MVC pattern so my code doesn’t get too mixed in with my UI.  Now I’m sure there are plenty of libraries and frameworks out there that would help me with this, but I’m rolling my own… if you can call it that.  I’m used to .Net’s code-behind method so I’m really just copying that to an extent.  Each php page will have an include at the top that references the code-behind.  The “code-behind” php file will set up all the data and will provide functions that can be called by the front one.  Simple, but it should be effective.

Finally, I’m using JQuery for UI prettiness.  I had a serious look at a lot of javascript libraries and JQuery came out on top for what I needed.

So that’s it.  That’s what I’m doing.  So far it’s all fitting together quite nicely, but I’ll be sure to post updates if I want to rave about something or if something sends me into a fit of frustration.

As always, comments are welcome.

Damian

So I have this project that I’m doing for my own interest and I constantly find myself getting nowhere with it.  I’ve done plenty of design work and I have my schemas and stories sketched all over the pages of a notebook, but there’s nothing really tangible electronically yet.

The most common excuse I give myself is lack of time, but I don’t think that’s really true.  After all, I have tons of design work done and you’d think that’d be the boring part right?  I think the problem is lining up spare time, motivation, and resources.

Spare time exists in snippets everywhere.  When I’m on the bus, in a particularly boring lecture, at home after work, etc.  Motivation is more fleeting.  Sometimes I’m really keen to get into this project, other times it’s the furthest thing from my mind.  Finally, resources are either available or they’re not.  I haven’t had a laptop for a while, so I intended to develop mainly from home.  I’ve borrowed a laptop, but I don’t want to fill it with all my development tools.  Having said that, I’m doing it in php and I don’t need much of an IDE to cut code.  I could probably get some meaningful stuff done in notepad if need be.  Testing? That’s harder, particularly as I’d need a database running and a php web server.

So you see that even though I’m motivated more often than not, those three things don’t always line up well.  I don’t often have much motivation when I get home from work, even though I have time and resources.  When I’m sitting in a boring lecture, I’ve got plenty of motivation and time, but I don’t have the resources.  You get the picture.

So, how do I change this?  Well you’ll be pleased to know I have a plan:

1. I bought a laptop.  It’ll arrive in about a week.  It’s powerful and I can put whatever the hell I want on that thing so you can rest assured it’ll be running all the dev tools I want.
2. Portable Apps.  I rediscovered these when I was thinking about how I could improve the situation.  If a computer is available, I can spin up a full WAMP stack as well as GIMP, Notepad++, and FileZilla.
3. Target deadlines.  I have them.  And work breakdowns, heirarchical todo lists and all that stuff.  I’m organised.

So when I wrote that post yesterday, to my surprise I found three other draft posts of varying ages.  I know myself well enough to know that I probably won’t go back and revisit them, so here’s what they were about:

Salling Clicker

That’s all that was in this one – just the title.  I know what I was going to say though.  I was going to heap praise on this tiny but personally very useful app.

I have a beautiful (expensive) Sony LCD TV in my living room with a very handy VGA port.  My computer is right nearby so it’s permanently connected.  What Salling Clicker does is provide a way to control my computer from my mobile phone via Bluetooth or Wifi.  This is amazingly useful when watching movies from the couch.

The interface is solid and deployment is very easy.  Best of all, it’s cheap.

Understanding threading is more important than ever

This was a post I was going to write about a now old Ars Technica article.  The article was about the future of processors according to Intel where they suggested there’d be many many cores, not just two or four.  I didn’t get too far on my post:

We were taught threading in the very first programming subject I took at uni.  Initially, it can be difficult to get your head around.  Eventually I got there.  I was convinced though that plenty of people passed that course without any firm idea of how threading worked.

With the announcement from Intel that they’re heading down the track of many, many CPU cores, understanding threading is more important than ever for a budding software developer.

What I was going to get to was that developers will have to start thinking about parallel processing.  If we’re stuck on 3GHz per core and you want to do some crazy-complex stuff, it better be able to be spread across the cores or it will be S.L.O.W.

Will the compiler/OS/processor be able to handle this for you?  Well, yeah, possibly to an extent, but writing your software to take advantage of new processor capabilities will be a necessity for performance.

At the very least, you’ll have to have a detailed knowledge of how threading works.  It’s the future, man.

Religious Coding

The oldest draft of the lot was also the closest to being finished.  Here’s what I wrote, minus the half-written final sentence (seriously, I stopped halfway through a sentence):

I stumbled across this Slashdot post the other day which talks about the potential for mobile phones to generate enough power to run by using piezoelectric materials.  In short, the vibrations you and those around you make when you talk could generate enough electricity to keep your phone going.

Now here’s the point in this post where I really wish I could remember the word I want to use.  You know when you hear about something for the first time in your life, then amazingly you hear about it several times in the next short while?  You wonder how you’re suddenly being bombarded with this new concept when you only just found out about it?  Yeah, there’s a word for that and I’ve forgotten what it is.  That word goes here.

I picked up a Popular Science magazine a few days before I found this Slashdot post and amazingly, there was a whole section in it about “free” energy.  Awesome concept.

There’s a whole host of low-powered devices in common use today, and there’s superfluous energy being expended everywhere.  Surely this energy can be used to power these low-powered devices?

The simplest example in the Popsci magazine was the humble light switch.  When you wire up a house, you have wire for power going to the actual light, as well as wire for control going from the light to the light switch.  When you flick the switch to turn the light on, you’re expending energy.  Not much, granted, put apparently enough to send a wireless signal to the light (or a controller next to it) to tell it to turn on.  With a setup like this, there’s no need for wire to run down the walls to the switches.  Yeah, it’s not much of a power saving and it’s probably not worth the extra initial cost right now, but in the future it could, and combining a few dozen simple ideas like this in every house could surely save some energy.

The fact is, we expend a lot of energy when we do anything.  There’s also ambient energy all over the place – the obvious ones like wind and light, but also vibrations and body heat.  If we could capture a fraction of that energy and put it to use, we’d have much less need for power plants.

There are thousands of options out there.  If you go for a walk, chuck your mobile phone in the little holster on your belt, and as you walk, the up-down motion of your steps can charge it.  Or maybe the energy absorbed when rain lands on your car roof could be enough to power your windscreen wipers.

Perhaps after a time, when it becomes cheap enough, device manufacturers may get into the habit of building these “free” power sources into their devices.  A mobile you didn’t have to plug into a wall or a sound-powered LED light you could just stick onto a ceiling would certainly have a market.

Clearly this stuff interests me.

So it’s been a long time since posts but I have an excuse.  Every time I thought about throwing up a new post, it occured to me that there’s only one thing I really want to write about, and I couldn’t.  Until now.

I’m moving on from my current job.

I won’t delve too deeply into the reasons, but the short version is that I’ve really done all I can for my career at Global MoneyLine at the moment.  I assume it’s the same with most financial institutions, but times are tough and the emphasis is understandably not on exciting new projects and new technologies.  Consequently, I’m not really learning new things and expanding myself any more and for me, that means its time to go.  It will turn around, sure, but I guess I’m just impatient.

So I start my new job with The Systems Works (www.tsw.com.au) on Monday next week.  Excited, anxious, slightly nervous, but very much looking forward to the change.  I’ll keep you posted.

A mate of mine working in the US sent me a link to a Mozilla Labs tool called Ubiquity.

Essentially, the idea behind it is to let users describe what they want to do with the Internet rather than where they want to go.  The web at the moment is really based around sites that you have to actually visit to be able to use the information.  Their idea is to skip this visiting stuff and let the tool (or commands written for it) do all the work for you, giving you only the information you were after.  They have a really good description on their blog post (linked above), and I can’t really think of a better way to describe it than them (which is probably good), so have a read if you’re interested.

It’s really lightweight and very easy to install, but at the moment it only works for Firefox.  It’s also surprisingly easy to write new commands.  I’ve been a .Net kid for a while now but it only took me a couple of hours to hack up a new command.  Admittedly the Ubiquity language is Javascript so it’s not terribly unfamiliar.  Still, that’s a pretty short learning curve.

Commands have the ability to give you a realtime ajax preview before you actually execute the command.  In most cases, I don’t even use the execute functionality and I suspect that for the small things, most people won’t.

Anyway, when I was trying it, the built-in “define” command wasn’t working for me.  It would look up a word when you pressed enter to execute the command, but it didn’t give me a preview.  So, in order to get a nicer dictionary lookup and at the same time try out this funky little thing, I decided to create my own.  Of course once I’d got mine working, the define command started working properly.  I like mine better – it gives prettier results.  It’s not just because I made it myself… but that’s the main reason.

You can go here to have a look at it.  Subscribe to it if you want to use it or hey, just steal the code or whatever, I don’t care, it’s public license – do what you want.

So in summary, it’s easy to use and I think it has the potential to be something really big.  If you have a look at the examples of what they’re ultimately trying to be able to do, it’s pretty impressive.  Imagine typing “find restaurants near me with reviews over 4 stars” and being given a google map with markers and summarised reviews of each.  Very useful, no?

Hmm, combine that with voice recognition and you’ve got a genuine futuristic computer from the movies!  Rad.

So I haven’t written a post in a while, but no, that’s not the reason for the title.  I’ve just been concentrating on other things and haven’t felt sufficiently motivated to write about anything.

But then I read this post from Steve Yegge on Stevey’s Blog Rants.  His title was “Business Requirements are Bullshit” which, while clearly designed to catch the eye, doesn’t entirely represent what it was about.

Steve’s post was aimed at people developing a new (or better) product to take to market.  He wasn’t talking to consultants or employees who are producing something to meet a specific company’s business needs, but someone who was creating something new to fill a perceived hole in the market.

His point (adapted from Warren Buffet) was that you should build something you already know about; something that actually meets your needs.  If you’re doing that, then you already know what you want.  You know what compromises you can make and what the unspoken and tacit deal-breakers are.  If you’re trying to gather business requirements from a group of people who may or may not want the product while trying to understand what it is you’re actually making, then you’re probably going to fail.

It sounds like great advice, but right now, I’m not in the category of people building something new to take to market.  I’m in the other group.  The stuff I build and maintain (and now I’m going to slip back into software) is supposed to meet an immediate business need for a specific client.  It more than likely won’t be used by me, but I have to build it anyway.

So can Steve’s advice be applied to my situation?  Sure it can, to an extent.

Steve’s main point was that if you’re not investing in something you understand, then you’re walking into very dangerous territory.  As an end-to-end software developer, I’m aware that it’s difficult to know exactly what the customer wants.  Sure, you can grill every potential user for days, you can write a comprehensive list of requirements, you’ll check it and recheck it over and over again to make sure you know single possible piece of functionality that they want and need.  But when it comes to the crunch, no matter how much work you’ve done, you’ll start getting negative feedback about some specific things that you hadn’t thought of and the customer hadn’t mentioned.

That complex report you were asked to include, the one with all the tables and forecasts and things?  It turns out that 10 different people print that 20 times a day.  So even though it’s perfect, it takes 15 minutes to run each time, and they can’t wait that long each time.

So how do you avoid this? In my experience, if you want to write truly useful software, you need to understand why each piece of functionality is being written.  Spend a lot of time with the clients and find out what it’s like to be in their shoes.  If you see how they operate day-to-day you’ll start to get an idea of what they actually want, not what’s written in the requirements doc.

Steve’s advice was that you shouldn’t invest in what you don’t understand.  So if you have to produce something you don’t understand, make a genuine effort to understand it first.  You might not be as enthusiastic about or deeply involved in the business your client is in.  But by honestly trying to see what they’re trying to achieve, you’ll learn what they really want your software to do for them; over and above the list of required functions.

So I discovered that Google Streetview is now available in Australia.

For example, here’s the Queen Street Mall in Brisbane.

I’m fairly blown away by the sheer number of streets that are covered to be honest.  This map gives you an idea, but once you get to the capital cities, you can see just how dense the coverage is.

On several occasions recently I’ve had to work with data in legacy applications.  It’s never fun, and it got me thinking about the origins of the problem.  It’s fairly obvious that difficulties encountered when working with legacy data are a result of decisions made when the system was originally written.

New systems are not immune, however.  The same difficulties are bound to crop up for someone trying to work with your data at a later date unless you think about it now.  There’s obviously no definite way of knowing how your data will be used in the future, and the tendency is often to focus only on current requirements.  With a little care though, you can make it easier for a future person to work with.

Thus, I present my guiding principles for data capturing.

By way of disclaimer, I should mention that these principles are based on business applications where users, be they staff or clients, enter information manually.  That’s not to say that they can’t apply in other scenarios, but keep in mind that I was looking at it from a business application angle.

So here they are:

1. Am I capturing everything I need?
2. Am I capturing information in a way that allows it to actually be used later?
3. Am I restricting myself – can I capture more (or different) information later if it’s needed?

1. Am I capturing everything I need?

If you’re not sure, capture it anyway.  For example, if you’re not sure whether the sex of a client is required, put it in anyway.  Obviously there are legal restrictions on collecting some information, but remember that these restrictions can change.  Allow for that.

More to the point, capture information with as much detail as you can.  Don’t hide behind a usability excuse – if you think hard enough, you can get around that.  Just because you store information in a particular way doesn’t mean the user has to enter it that way or even enter it at all – some information can be assumed.  Of course there’s always some trade-off with usability, but it’s worth thinking about how you can get around the usability problem rather than leaving a potentially important piece of information out.

For example, timestamps on file comments probably don’t need to be accurate right down to the tick, but what’s the harm if they are?  Let’s face it, storage space is cheap.  In terms of hiding behind usability, your user shouldn’t have to enter a time to that detail (or probably at all), but it doesn’t mean an automatic process you write at a later date isn’t able to.

2. Am I capturing information in a way that allows it to actually be used later?

This seems obvious but is often overlooked.  Don’t store dates (or any other numeric information) as strings, don’t store information in binary fields, and use a database not a file to store information.  If your system takes a piece of information from somewhere, be it a person or another system, it should store that information in a way that can be used easily later.  There’s no point retaining information if it can’t be used.

Let’s say the business wants you to store a whole lot of information about a client’s car.  The designer might give the user fields to fill in the licence plate, the colour, make, model, VIN, age, etc.  The easiest way to store most of these pieces of information is in string fields.  Sure, this will allow users to look at the data, probably interpret it, and use it to glean information, but what if you need to analyse it in bulk later?  You’ll have a lot of work to do to group all the “White”, “white”, “wite”, and even “wh9ite” cars.  It’s hard to work out that cars that are “12mths”, “12 months”, “1Y”, and “one yr” old are all the same age.  Don’t do it to yourself.

Typically, the designer might ask of the business, “Are you ever going to search on this data?” That’s a mistake.  Don’t even ask.  They will say no.  However, some years down the track, they might actually want to know whether there’s any correlation between having a new car, and being a good client.  It may seem far-fetched now, but by allowing for it, you don’t take it off the table.  The business should never hear, “I can’t give you that information” when they know for a fact their staff have been filling it out every time.

It all comes down to working out what the information actually is and what it means, then storing it the same way.  If it’s a length of time, work out how to store it as a timespan.  If it’s a colour, try working out how to actually store it as a colour, not someone’s misspelt description of a colour. Do the research and find out how a VIN or a licence plate is structured so you can validate it.  If it’s really not possible to validate, at least process it as much as you can.  Remove spaces or capitalize all letters if it doesn’t matter to make it as easy as possible to work with at a later date.

3.  Am I restricting myself – can I capture more (or different) information later if it’s needed?

Don’t save space in fields when it could come back to bite you. Don’t use a 4-bit integer to save space because you know you’ll never have more than 16 different payment methods or vehicle makes.  Even though you’re only operating in Australia, use string encoding that allows Chinese characters.  What are you saving by skimping, really?  Again, storage is cheap.  Don’t restrict yourself from the start.

If you have to use your judgement to determine the size of a field, then overestimate significantly.  Like, really a lot.  Can’t imagine someone having a surname longer than 30 characters?  Me neither, but I’ll allow 100 or more.  I’m not having some client with a four-way-hyphenated-surname break my application.

Summary

Don’t take the easy way out.  I know if you’re stressed for time and under the pump, adding a new Bank Name field as an non-validated string is going to be a lot easier than trying to match it to a list of known banks (with the option to add a new one).  Fight the urge to make it easy now and potentially very hard later.

Remember that your design decision is usually a one-way street.  It’s not often that you can do it the easy way now and fix it up later.  Even if you can, will you?  Really?  You’re too busy to deal with it now, what makes you think you’ll have plenty of spare time to do it later?  It’s worth the extra time to do it properly.  You’ll thank yourself later.

In a previous post, I talked about the limitations with CAPTCHA systems and proposed a partially-automated turing test to keep non-humans out.

I had a few discussions about this with a friend and he was more interested in the CG Kittens idea I very briefly alluded to in the main text. To summarise, the idea was that if the kittens in KittenAuth were 3D models rendered on the fly, you’d get an essentially unlimited number of images.

Well someone has gone and done something very similar. On a small site called SpamFizzle, there’s a description of a 3D CAPTCHA design that renders simple objects and asks the user questions about them. It looks like a great idea to me. It could require some fine-tuning, but I think the premise is sound.

I’d love to credit the author some more, but there are no details on that page. I presume it’s the same author as the only other page on the site, Michael G. Kaplan.

Regardless, it’s a great idea and worth a read if you’re interested.