More photos from the garden

My wife also found this Privet Hawk Moth caterpillar (Sphinx ligustri) munching away in our garden. Apparently it's the biggest moth we get here in the UK, the moths can have a wingspan of nearly 5 inches (about 12 cm). It was hanging upside down on our privet hedge:

I think it was reasonably young, because I reckon it will get even bigger. It was great to find it and be able to take a couple of photos.

Butterflies in the garden

The buddleia bushes we planted in the garden a couple of years back have started to get really big now, so this year our garden has been full of butterflies. I thought that I'd share a couple of the snaps I took:

I'm not a very good photographer, but you get the general idea. I've found it quite relaxing to enjoy a few moments on a sunny day with all the butterflies flitting around.

Gold Rush

I've become hooked on the TV series Gold Rush which comes out on the Discovery Channel.

In fact, I seem to watch a lot on the Discovery Channel these days. I blame Mythbusters. They started it.

But I suppose that any program involving a load of big digging machines combined with the thought of being able to extract gold from under your feet is bound to come out pretty good...

Having said that, I do find myself shouting "you're an idiot" at the TV quite a bit. So it's probably one of those things where you'd like to think that you could do better yourself. In reality, I'd probably not last five minutes if you sent me to Alaska and told me to try and do what they're doing.

I think that part of the appeal is the often makeshift way they go about fixing things or solving problems. It's all very man-with-beard-in-his-shed type stuff, and I think we could do with more crazy men with beards building things in their sheds. I'd like to be one myself - but I'm more of a man-with-stubble-working-in-the-spare-room. But I'm doing the best I can, okay.

Iced latte saved my life

Well, two iced lattes actually. Yesterday, I was presenting at an implementation workshop in Westminster. It was the hottest day for 7 years apparently. It was over 32°C in London. Just riding on the tube was enough to make you sweat. It really gets hot down there.

So when I got near the venue, I went looking for some kind of cold coffee type beverage. I ended up just round the corner, in Victoria ... where I found a Coffee Republic.

[Photo taken from google street view]

So I went in and bought an iced latte - and didn't regret it. Then, in the afternoon, when my presentation was over (which was in a non air-conditioned room) and I was baking hot, I went back to Coffee Republic for another one.

I then journeyed back down into the tube, with my iced latte and the ice cubes still rattling round inside the cup. There were many envious (even desperate) glances from other passengers as I sipped on my ice cold coffee. Sorry guys.

So, if you're in the Westminster/Victoria area of London, try the Coffee Republic down there. The coffee was great and the service came with a smile too. I expect I'll be going back next time...

Self configuring embedded database in C#

I recently needed to write a C# desktop application which used its own small database. I wanted something that could be easily deployed with xcopy, so I didn’t want to connect to any external database engine. So I used SQL Compact Edition v4. Since I’ve recently been having success with PetaPoco as a micro-ORM type thing, I thought that I’d try that too. I had the idea that the database could configure itself when the program starts, so that’s what I set out to try. I was surprised how easy it was. This is what I did to make a proof-of-concept:

  • Create an empty Console Application in Visual Studio 2012
  • Use NuGet to add references to PetaPoco.Core (v5.0.1) and Microsoft.SqlServer.Compact (v 4.0.8876.1)
  • Have this code as the C# program

That’s all there is to it, easy. If you corrupt the database, then the code will re-create a clean database file next time it runs. You don’t need to include an sdf file in your project, because the database will be created by the code. Obviously this is an extremely trivial example, but it seems to work great.

Creating self-signed test certificates with Bouncy Castle

I've been messing about with Bouncy Castle as a means to create self-signed X509 certificates for testing purposes. Essentially I've taken the information from these three blog entries here and here and here, making them into one piece of code. It seems to be a pretty nifty way to make test certificates using C#.

Here are some instructions: first, create a new Console Application in Visual Studio (I'm doing .Net 4) and use NuGet to add Bouncy Castle (I've got v1.7.0) to the project. Then simply use the attached code example. It demonstrates creating a fictional type of security certificate and saving it to disk.

Obviously, I can't take the credit. The clever stuff is all done by Bouncy Castle, and Roger's Blog has demonstrated how to write the C# code. All that I've done is put it together in one place.

The result is a .net X509Certificate2 object which can be used in the normal way.

Random inspiration from Accelerated Massive Parallelism

For some reason I've started looking at the sample code for Microsoft's C++ AMP: Accelerated Massive Parallelism. The temptation to offload work onto the GPU has finally proved too much. Actually, I've looked at various ways to do that in the past, but then slowly walked away...

This time I got as far as calculating the Mandelbrot set by means of C++ AMP, but I won't bore you with that, and besides, Microsoft have provided their own sample code and it's probably better than my version. But when I looked that their code for calling C++ AMP from C# code I was reminded how simple it is to call C++ code from .Net, and I started to wonder why I don't do this more often - even without any Accelerated Massive Parallelism trickery. For example, say you're doing some repetitive calculations, why not do the maths in a small bit of C++ code and get a huge speed improvement? It's not that difficult to do a simple calculation in C++ and you don't need lots more code.

So I spent a lunchbreak writing a sample project to calculate the biggest distance found between an array of points (x,y coordinates). I implemented the same calculation in both C# and C++ and then ran it on an array of 10,000 random points. Using the C++ code, without any help from AMP, it was about ten times faster - even thought I was calling it from within my C# console app:

Wow. That's quite a big difference. Note: I'm computing the distance between every combination of two points in the array and returning the biggest distance found.

Now ... I may not be able to do this type of stuff during my day job, where everything needs to be done in C# only, but I think that I may be tempted to do this type of stuff in my own time when I'm just messing around.

Testing for memory leaks in C

Whilst wandering round the Cambridge University bookshop the other weekend, I noticed a book called Memory as a Programming Concept in C and C++. It was just too tempting, so I bought a copy.

Nobody has ever taught me about memory management in C, I've worked this stuff out from experience and probably by reading other peoples code. So it's nice to read a tutorial - even if I've already been doing this stuff for years. If nothing else, it's made me think more carefully before I dynamically allocate some memory. And I thought that I was pretty careful in the first place.

So I decided to write some debug code to test for memory leaks in my H2D2 virtual machine. I was pretty confident I would not find any problems. But, I did actually find a calloc() where I had meant to put a realloc(). A silly mistake where I wasn't being careful enough. This was leaking memory, because I had a malloc() which was being followed by a calloc() instead of realloc() - so the memory allocated in the first place was not getting freed up (only the memory from the calloc() was getting freed up).

Here is an example of the debug code that I've written: view C code (as a text file).

You just need to replace your calls to malloc, calloc, realloc, and free with rmalloc, rcalloc, rrealloc, and rfree. As you can see from the code, it simply maintains an array of pointers to dynamically allocated memory, so when your code exits you can see if there are any 'dangling' pointers left over. Simple, but effective. If there are no pointers left when your code finishes then you're probably not leaking memory. Nice.

Anyway, I'd be happy to recommend the book 'Memory as a Programming Concept in C and C++', I've found it a useful recap on many things and a good reminder to be extra careful with dynamic memory. It's also been fun to write some memory debugging code, which is a tangent that I may not have gone down if I had not bought the book. Although this book was published 9 years ago, it's nice to see a good book on C programming that's less than a decade old. Thanks Frantisek!

Some H2D2 progress, whilst messing with Run Length Encoding

So progress on my programming language, still codenamed H2D2 continues on background threads :-) I've not given up, but I haven't made many posts on the blog about it recently. It was probably getting boring anyway, and recent work has been tidying things up and writing unit tests - which does not make for interesting reading.

One thing that I wanted to try, was to compress the H2D2 bytecode so that it doesn't take as much time to serialise a running program between machines. I believe that Java JAR files are zipped, so I thought that I'd do something similar with H2D2. But I also wanted something that I could write myself that didn't take an age. Since I'm doing this for fun, it's nice to start with a blank page and go from there. So anyway, my web wanderings brought me to here. That page talks about Run Length Encoding - something that I implemented once myself back in the 1980s (or maybe the early 90s) - not that I have the source code anymore. But I read about the RLE PackBits algorithm and thought it might be a good way to go. Not a massive processing overhead and reasonably simple to implement. I chose not to look at the example souce code provided ... and just to write it from scratch myself. It sounded like fun.

It took me about an hour whilst drinking my morning coffee to write the compression code, and an hour or so in the evening to decompress the data back to the original values (I also wrote some unit tests to check for any obvious problems I could see). It seems to work OK: when I compress H2D2 bytecode I'm getting about a 30% reduction. It's not as good as zip compression obviously, but it will do for now.

I'm sure my code won't win any prizes for optimisation or style, but at least it works. I guess it might also come in handy if you were trying to squash some data on a microprocessor without needing a massive library or lots of processing power. So here is an example C program:

// Example RLE compression with PackBits
// ...just an example, not production code :-)

#include <stdio.h>
#include <string.h> // for memset

void fgrow(char*, char*);
void fshrink(char*, char*);

int main(int argc, char *argv[])
{
  if (argc==4 && strlen(argv[1])==2 && argv[1][0]=='/')
  {
    switch (argv[1][1])
    {
      case 'c':
        printf("Compressing... ");
        fshrink(argv[2], argv[3]);
        printf("done!\r\n");
        return 0;

      case 'u':
        printf("Uncompressing... ");
        fgrow(argv[2], argv[3]);
        printf("done!\r\n");
        return 0;
    }
  }
  
  printf("Usage: RLE { /c | /u }  \r\n");
  printf("   use the /c switch to compress or the");
  printf(" /u switch to uncompress\r\n");
  return 1;
}

void fgrow(char* inpath, char* outpath)
{
  char ch=0, hdr=0;
  FILE* op = fopen(outpath, "wb");
  FILE* mp = fopen(inpath, "rb");

  while (!feof(mp))
  {
    hdr=fgetc(mp);
    if (feof(mp)) break;

    if (hdr>=0) // copy bytes verbatim
    {
      for (; hdr>=0; hdr--)
      {
        ch = fgetc(mp);
        fputc(ch, op);
      }
    }
    else // copy a run of bytes
    {
      ch=fgetc(mp);
      for (; hdr<=0; hdr++)
      {
        fputc(ch, op);
      }
    }
  }
  fclose(mp);
  fclose(op);
}

void fshrink(char* inpath, char* outpath)
{
  char buf[128];
  memset(buf, 0, 128);
  int n=0, hdr=0, runlength=0;
  FILE* op = fopen(outpath, "wb");
  FILE *mp = fopen(inpath, "rb");

  while (!feof(mp))
  {
    buf[n++] = fgetc(mp);
    if (feof(mp))
    {
      n--;
      break;
    }

    if (n==128)
    {
      // buffer is full
      hdr=127;
      fputc(hdr, op);
      fwrite(buf, 1, 128, op);
      memset(buf, 0, 128);
      n=0;
      continue;
    }

    if (n >= 3 && buf[n-1]==buf[n-2] && buf[n-2]==buf[n-3])
    {
      // last three bytes are the same
      hdr = n-4;
      char symbol = buf[n-1];
      char tmp = symbol;
      if (n > 3)
      {
        fputc(hdr, op);
        fwrite(buf, 1, n-3, op);
      }
      runlength=3;

      while(!feof(mp) && tmp==symbol)
      {
        tmp = fgetc(mp);
        n++;
        runlength++;
      }
      hdr = 2-runlength;
      fputc(hdr, op);
      fputc(symbol, op);

      // put the different byte back for next read
      if (!feof(mp)) ungetc(tmp, mp);
      memset(buf, 0, 128); 
      n=0;
    }
  }

  // flush the buffer if not empty...
  if (n > 0)
  {
    hdr = n-1;
    fputc(hdr, op);
    fwrite(buf, 1, n, op);
  }

  fclose(mp);
  fclose(op);
}

Obviously this compression will only shrink files which have runs of the same repeated byte value in them. Otherwise it may increase the file size. You can try it from the command line and pass /c as the first argument to compress a file and then use /u to decompress the file back. The next two arguments need to be the names of the input and output files respectively. This example doesn't have any error trapping, so if the file fails to open, or something like that it will go bang. But that would not be hard to add.

Of course, in my H2D2 code, I'm using the same algorithm, but I've adapted the code to work on blocks of memory rather than on files. But in H2D2 I have made a VFILE struct to work alongside the C FILE pointers and I've added some functions that work the same as the C IO functions but taking my VFILE pointer instead. This means that I can easily swap code to operate either on disk or in memory...

Honda Unhappiness

If you're considering buying a Honda used car from a main dealer or are thinking about extending your Honda warranty, you may want to read my recent experience...

I've long been a fan of Honda cars. I've owned many, and most of them have been very reliable. However, the CR-V I bought in December last year developed a serious engine fault, so bad in fact that the dealer decided that the engine could not be repaired. The dealer said the whole engine needed to be replaced.

The car was fully covered by the 'Honda Happiness' warranty, but unfortunately, the dealer wanted to do the repairs at their own expense, and not use the warranty. This left me without a car. The warranty would have given me a hire car, but since the dealer was not making a claim on the warranty I found myself off the road. I was definitely not experiencing 'Honda Happiness' when I found myself taking the bus to work, to say the least.

After this dragged on for a week, and with no end in sight, I asked the dealer to just give me my money back. So I had to go and find another car to get back on the road again. But this seemed to be the quickest way to get my transport back.

Needless to say, I complained to the 'Honda Happiness' administration people, because the warranty that came with the car proved useless to me. But they've still not helped. I've taken it as far as I can, and this is their final response:

So, Honda are saying they've done nothing wrong. Meanwhile, the dealer said they're not obliged to use the warranty. And it seems that the only person who has lost out is me, the poor old customer. I would say that this is one of the worst customer service experiences I have ever had.

Ultimately, it seems that the 'Honda Happiness' warranty only helps you at the dealers discretion. Even if your car has a very serious failure like mine, it doesn't mean you're guaranteed to get any help. And if things go wrong, even if you complain, nothing seems to change. Even a token gesture to acknowledge all the inconvenience would have been something.

So, I'm left very disappointed indeed. Until now, I have considered myself a Honda enthusiast. As you can imagine, nowadays - not so much.