For the last few months I’ve been working on a project that is part website and part data processing.

In theory the website is the bulk of the project, but in practice processing the data has taken a disproportionate amount of resources.

Data exchange with another organisation is always tricky - and in this case we have incoming data that doesn’t conform to specification, business rules that change, and a daily data import to run.

The code I’ve been working on has been through a couple of versions, and now I have in mind a third