BMW functionality

(Part of an occasional series on matters technological. It beats doing real work.)

Before we started Matchpeg, we used to be Terribly Important People. We used to think nothing of donning our superhero garb in the nearest phonebox, and then leaping into a presentation with senior executives to sell them hundreds of thousands of pounds/dollars/euros of software and services at a time.

Like everyone else, we built our fair share of BMW functionality - not a reflection on the marque, but a comment on the sort of cars driven by the sort of people we were presenting to.

BMW functionality looks very pretty - albeit in a standardised, Stock-Option Chic kind of way - and it's not totally pointless. (A typical example is a ray-traced pie chart representing three numbers so simple that their relative proportions could be visualised, without graphical aids, by a child of seven.)

You're in a meeting where you're presenting the BMW functionality. You know that no one will ever actually "use" it. Even the users know that they'll never actually use it. But they're not in the room. You're presenting to the company's senior executives, and an 80/20 rule applies: they make 80% of the purchasing decision, but they're going to clock up at most of 20% of the eventual usage of the system.

At best, this is neutral: the BMW functionality is, in effect, part of your marketing (not your software); it helps to sell a fundamentally good system which deserves to do well; and it doesn't have any adverse impact on the quality of the system.

At worst, it's actively harmful: the time spent on the BMW functionality could and should have been spent on making the actual software better; and/or other, useful features are discarded or compromised in order to squeeze in what's effectively your sales pitch which the lowly users then have to live with day-by-day for the next n years.

This is less common on the web, because relatively few web-based applications are sold face-to-face. But there's still pressure to make sure that your software produces a great screenshot, and there's often tension between what produces the best screenshot and what produces the best functionality. (As an innocuous version of the same phenomenon, count the number of web-based applications which are demonstrated using Mac screenshots despite the fact that 97% of their users will be running Windows. Yes, we do this. Everyone does. It's the harmless end of something much more serious.)

What does all this lead up to?

We took a vow when starting Matchpeg: no BMW functionality. Deliver results.

NEW feature - notes on workflow items owned by someone else

Bowing to very vocal demand, we've added a new feature to the workflow system: the ability to add a note to a workflow item which you don't currently own. (It continues to be the case that the current owner is the only person who can change the workflow item in any way.)

The ability to add these notes is controlled by the workflow definition. This switch is turned off for all existing workflows. If you want to take advantage of this new feature, you need to edit your existing workflow definitions and turn the switch on.

After doing so, anyone who's not the current owner of a workflow item is able to add a note to it using the new box at the bottom of the item overview page.

Project management

(Part of an occasional series on matters technological. It beats doing real work.)

We've been having some internal wrangling here at Matchpeg Towers about project management/software development methodologies (not much wrangling - we're all fugitives from organisations which like signatures in triplicate on everything).

Matchpeg is essentially in the retail software business. We sell tools at low cost, in large quantities, to happy customers who we rarely have any un-automated contact with. We do very little bespoke work. We don't really have "clients".

There's a polarised debate all over the web about the relative merits of waterfall (relatively linear, relatively traditional) software development processes versus more agile methodologies.

Waterfall doesn't work for us. It's not that we're a bunch of laid-back hippies - agile doesn't work for us either.

The problem with both is most concisely represented by the Agile Manifesto. Notice anything about this? (Leaving aside the fact that it reads like a Miss World contestant's pledge to seek out world peace and a cure for cancer.)

The references to customers are both in the singular, not to "customers" plural. We're in the retail software business. We have lots of customers. And they're so diverse that you can't boil them down into a single representative entity called "the customer" (or even "The Customer").

(The same failings can be found in things like the official PRINCE2 material: same assumption of a single customer; same assumption of a single entity paying for the project; same assumption that there's a contract.)

In short, our experience is that no well-known software development process translates at all well to the retail world. They all seem to stem from a founder's experience of in-house development for a large company, or bespoke development for a single (usually large) client. Using a popular term in IT, these methodologies don't "scale" well. They make all sorts of assumptions which stop working when you're dealing with a number of diverse customers, each of whom isn't guaranteeing to pay you anything.

These methodologies are created by consultants in order to perpetrate consultancy. They're for use with "clients", not "customers".

--

On a related note, we're bemused by the way the debate between waterfall and agile is usually presented: an either/or choice, generally accompanied by comments such as "waterfall is so lame".

Instead, it seems to be about risk/reward profiles. A question such as "Which is better, waterfall or agile?" looks as meaningless as "Which is better, stocks or bonds?" The answer depends on your circumstances and what you're trying to achieve.

As a very general rule - to which there are, yes, countless exceptions - the anecdotal reports on the web tend to indicate that agile methodologies offer the prospect of better returns (faster development) at the risk of greater loss (complete anarchic breakdown of a project into marauding bands of developers).

It's not a coincidence that waterfall methods tend to be preferred by larger, older firms, and agile tends to be preferred by smaller, newer firms.

If you're a small company just starting up, you have to take risks. You gravitate towards agile because you're not going to get anywhere by being dull-but-steady. Older companies - particularly public companies - don't and can't have the same appetite for risk. Everything militates against it: the shareholders want predictable returns, the employees want to keep their well-paid jobs. The potential additional reward doesn't justify the additional risk; the reward is too incremental, and the risk is too absolute.

The really nasty fights start when the incentives and risk-tolerance of the business owners are different to the incentives and risk-tolerance of the people running projects for them. This works both ways: an impetuous CEO who finds that he's hired a pen-pushing dullard of a project manager, or a cautious board which has landed itself with a bunch of mavericks who won't be significantly affected if they cause their project (or even the entire firm) to implode.

NEW feature - removing brainstorming suggestions in real-time

We've added a new feature to brainstorming suggestions, particularly designed for use when running sessions in real-time. In addition to the full transcript editor, the organiser of a session can now delete ideas from the session, during the brainstorming stage, by clicking on the new grey cross icon which is displayed to the right of each idea.

This simply deletes the idea from the session. It has exactly the same effect as using the transcript editor to remove the idea. Like the transcript editor, only the session organiser is allowed to do this (and only they get given the icon for doing so.)

NEW feature - user groups

We've finally added a feature which we'd had in mind for a long time, and which several customers then prompted us by asking for...

You can now define a group of users (i.e. a list), and use it as part of a workflow definition. For example, you can define a "Customer Services" team, and have customer enquiries and requests being allocated to that team rather than to a specific individual.

It remains the case that every workflow item is always assigned to one specific person: if the workflow definition says to assign the item to a group, the system picks an individual from the group and assigns the item to them.

The definition of a user group lets you define how work is shared out amongst the group members: simple round-robin allocation of tasks, allocation based on who has the fewest items in their workflow queue etc.

You define user groups from the user list - i.e. click on "User list" from the Dashboard, and then click on "User groups". Only administrators are allowed to change (or delete) groups.

Groups can then be used anywhere within the workflow system where it asks who to assign a workflow item to.

As ever, full information can be found in the help file. We particularly recommend looking at the section entitled "Using groups as aliases".

Does Matchpeg support the Opera web browser?

Matchpeg supports Internet Explorer (v6 or later), Safari, and all Gecko-based browsers such as Firefox and Camino.

We don't support Opera.

This is because there's an obscure bug in Opera which affects only Matchpeg and a few other sites. The main effect is that error messages don't get displayed - if an error occurs, we have to display a generic failure message rather than the proper explanation of the error. (See below for the technical reason why.)

So, Matchpeg is more or less usable with Opera, but if an error occurs you won't know what or why.

Yes, we've told Opera about this. No, they haven't done anything about it (or, indeed, even responded).

--

Tech details:

The problem is in Opera's XmlHttp handling. Opera doesn't follow the de facto standard which all other browsers adhere to. It's arguable that Opera's implementation doesn't even follow RFC 2616 properly.

When Matchpeg's web pages make XmlHttp calls (e.g. in order to log on), there are obviously two possible results: success (and some data), or an error. The XmlHttp pages on the server signal which has occurred using the HTTP status code. The usual code 200 means that the call has succeeded (e.g. logged on successfully), and the XmlHttp responseText member then contains any applicable data. The code 299 is used to mean that an error occurred (e.g. invalid user name or password), and responseText contains a textual error message to display to the user.

This gives the Javascript in the client-side web pages a simple way of checking what's happened. If you look at Matchpeg's Javascript, you'll see lots of code along the lines of the following:

switch (XmlHttp.status) {
   case 200:
       // Success. Do something with the
       // data in XmlHttp.responseText
       break;
   case 299:
       // Error! Display the message to the user.
       alert(XmlHttp.responseText);
       break;
   default:
       // Must be a 400+ or 500+ error - i.e. a
       // problem with the server configuration.
       break;
}

(We think this is quite elegant.)

The problem is that Opera drops the HTTP response body for all XmlHttp calls where the status code is anything other than 200. Therefore, a blank error message gets displayed.

There are three reasons for considering this a bug in Opera:

• No other browser behaves this way.
• There's nothing in RFC 2616 which obviously supports Opera's behaviour.
• If Opera makes a normal (i.e. non-XmlHttp) request to a page which returns a status code such as 299, then it does display the response body. In other words, Opera's behaviour isn't consistent, and this tends to suggest that the XmlHttp implementation is in breach of RFC 2616.

We could, of course, change Matchpeg to pass back XmlHttp results in a way which works with Opera. But, frankly, it's not worth working round a bug in a browser which about 0.3% of our potential customers use.

Matchpeg Sudoku

We're often asked, "Why does Matchpeg have a Sudoku solver?" And also, though not usually by the same people, "How does it work?"

Important information for non-Brits: it is compulsory for every British newspaper to print at least one Sudoku puzzle daily. The freesheets for London commuters typically have three (which gives you an insight into the average journey time of a commuter into London, and the fact that these freesheets are only distantly related to "newspapers", and have rather more in common with the puzzle compendiums which you can buy in any airport before undertaking a seventeen hour flight in steerage).

More useful background information: British newspapers are well known for their bouts of collective mania. Their peerless executives reach a more-or-less simultaneous decision to fill them with spot-the-ball competitions, bingo, vouchers for free flights and meals in restaurants, free DVDs. Sudoku was the last-but-one or -two of these crazes.

The creation of Matcheg Sudoku was prompted by The Guardian printing one of these witless puzzles on every single page of its G2 supplement. This spoiled a perfectly nice breakfast, and writing a tool to solve them provided a handy little intellectual challenge to kick off the working day.

We stuck the solver on the site because we feel it chimes quite nicely with Matchpeg's aim of solving business frustrations.

--

How it works.

From a programming perspective, solving a Sudoku puzzle is mainly a simple exercise in recursion. The best method seems to be to keep track of all the values which a particular cell can't be. For example, if the cell in the top-left corner is known to contain 5, then the twenty other cells in the first row, first column, and first block can't contain 5.

Each time you set a known value, you record these 20 knock-on values. And each time you record a value which a cell can't be, you do two checks:

• Does the cell now have eight values which it can't be? If so, it must be the ninth.
• Is it now known that eight of the cells in a row, column, or block can't be a particular value? If so, then the ninth cell must contain that value.

When you discover a new value as a result of either of the above, you fill it in and process the 20 new implications of that. So, setting a single cell triggers recursive calls setting the values of other cells.

This method by itself solves the vast majority of Sudoku puzzles - typically, everything other than those rated as "very hard".

The next step involves looking for cells in a row, column, or block where only two of the cells can contain a pair of values: for example, there are only two cells which can contain 4, and the same two cells are also the only ones which can contain 8. If so, then you know that those two cells must contain either 4 or 8, and you can eliminate anything else on their can't-be list.

We've never seen a puzzle where this extra step actually gives a solution rather than just narrowing things down a bit. You could apply further subtle logic to the solution, but that's more computationally expensive than simply brute-forcing the puzzle from this point onwards. In fact, the check for pairs usually takes longer than its benefits merit: it's quicker to skip this check, and just brute-force the solution without first looking for pairs.

So, if the Matchpeg solver can't work out the puzzle through simple logic, it starts guessing. This falls into two parts: trying out guesses for single cells in isolation, and and then trying out a combination of guesses for multiple cells if necessary.

The first part simply consists of going down the grid, taking every value which each individual unknown cell could still be, and seeing what happens if that value is filled in. There are three possible results from trying out each one of these isolated hypotheses:

• The value can't be right. Its knock-on effects are that the grid must be invalid (i.e. duplicate values in a row, column, or block).
• Filling in that one value solves the puzzle. In other words, the hypothesis becomes a lucky guess.
• It makes no difference. Filling in the value neither solves the puzzle nor is impossible.

Recording the hypotheses which can't be right (and their knock-on effects) usually means that you then hit a "lucky guess" very quickly. You typically only need to try out about 10 hypotheses before you get a solution.

In a very, very small number of cases, no single cell by itself gives the solution. You have to try out a combination of guesses for multiple cells at once.

The Matchpeg solver builds a list of the hardest remaining cells, and then tries all the possible combinations of the two hardest cells. If that doesn't work, it tries all the permuations of the three hardest. And then four, and then five.

However, we've never seen a puzzle which requires guessing four or five cells at once. The "world's hardest Sudoku puzzle" gets solved from the combination of guesses for only three cells.

NEW feature - central list of RSS feeds

As we hope you already know, Matchpeg makes a very large number of RSS feeds available. For example:

• Feed of all your pending meetings
• Feed of all your incomplete action points
• Feed of all your pending brainstorming sessions
• Feed of all your current workflow items
• ...Plus individual feeds for each meeting, workflow item, and brainstorming session

However, no web browser has quite cracked the issue of advertising feed-availability to people. Therefore, in order to make them more prominent and better-known, we've added a central list of the main feeds. This is accessible using the new "RSS feeds" link on the right of the Dashboard.

(If you have no idea what RSS is what are you doing reading a blog? we suggest that you read a generic introduction to the subject, such as the one published by the BBC. All Matchpeg's feeds can be added to places such as a Google home page.)

NEW feature - full help file

Matchpeg's context-sensitive help file is now complete - i.e. full documentation of the system in addition to the Getting Started guides, the brief help text displayed on each screen (e.g. at the top of boxes), and all the various tooltips.

Now that the help file is complete, help on each page can be viewed in either or two ways:

• Click on the new help icon in the top right of each page


• Or just press F1

The "Help" link in the "Help and support" section on the Dashboard continues to link to the first page of the help file.

We recommend that everyone - though particularly new users - reads the "hints and tips" section of the help file. Almost everyone will already know 80% of the information in this, but it will be a different 80% for each person, and the remaining 20% may well turn out to be valuable information.

Introducing the Matchpeg Introducer scheme

Big drum roll for the launch of... the Matchpeg Introducer scheme.

The basis for this is simple, and it's worked really well in our previous ventures: word-of-mouth recommendations are much more effective and much cheaper than any other form of marketing, and anyone who recommends us to their friends and colleagues deserves a share of the proceeds.

• You sign up as an introducer.
• You put a link to Matchpeg on your website/blog.
• People sign up with Matchpeg after coming to us via your link.
• We pay you commission whenever your referrals spend money with us.

We also provide other kinds of marketing support such as e-mail templates which you can use when contacting your friends and colleagues about Matchpeg.

You can even introduce other introducers: if someone becomes an introducer themselves after coming to Matchpeg via you, then you can earn extra commission on their referrals. (No, this is not a Ponzi scheme. Payments to introducers don't come from payments by other introducers. Registration as an introducer is free.)