Lenovo IdeaCentre K430 Desktop PC

So my wife needs to run a bunch of different Virtual Machines for her work. Doing that means she can easily clone an existing VM, mess about with stuff and throw it away afterwards if needs be. Then she doesn't need to worry about having left her computer with lots of weird settings at the end of the day.

So she really needed a fast machine with loads of storage for all those VM virtual disk images, and a decent amount of RAM as well.

We looked around, and she went for the Lenovo IdeaCentre K430. It has a 4TB hard disk, a intel Core i7 processor and 12Gb RAM. So there's lots of space for VM disk images there and a nice amount of RAM to run them. We've created a separate partition just for the VMs to be stored on. All that RAM means you can easily dedicate 4Gb of memory to a VM without any worries.

When I got my hands on the machine, there was loads of rubbish pre-installed by Lenovo, so I erased the entire disk, set up my own set of partitions and reinstalled from scratch. But it was a pretty smooth process. I didn't even need the Lenovo driver disk, everything seems to have been detected by Windows 8 automatically.

I was tempted to suggest she tried Linux as the host Operating System, but in the end I decided she'd be happier with Windows.

Using VirtualBox, when you set your VM to run full screen it seems just as fast as the old PC running natively, you don't really notice. So that's pretty good. But I will upgrade to Windows 8.1 as the host OS when it's released, I've been using the Windows 8.1 preview on my tablet and I like the tweaks that have been made.

... and I've virtualised the old PC so I can keep it as a VM, just in case there's something left on it that we need. Yet again Disk2VHD comes to the rescue.

More photos from the garden

My wife also found this Privet Hawk Moth caterpillar (Sphinx ligustri) munching away in our garden. Apparently it's the biggest moth we get here in the UK, the moths can have a wingspan of nearly 5 inches (about 12 cm). It was hanging upside down on our privet hedge:

I think it was reasonably young, because I reckon it will get even bigger. It was great to find it and be able to take a couple of photos.

Butterflies in the garden

The buddleia bushes we planted in the garden a couple of years back have started to get really big now, so this year our garden has been full of butterflies. I thought that I'd share a couple of the snaps I took:

I'm not a very good photographer, but you get the general idea. I've found it quite relaxing to enjoy a few moments on a sunny day with all the butterflies flitting around.

Gold Rush

I've become hooked on the TV series Gold Rush which comes out on the Discovery Channel.

In fact, I seem to watch a lot on the Discovery Channel these days. I blame Mythbusters. They started it.

But I suppose that any program involving a load of big digging machines combined with the thought of being able to extract gold from under your feet is bound to come out pretty good...

Having said that, I do find myself shouting "you're an idiot" at the TV quite a bit. So it's probably one of those things where you'd like to think that you could do better yourself. In reality, I'd probably not last five minutes if you sent me to Alaska and told me to try and do what they're doing.

I think that part of the appeal is the often makeshift way they go about fixing things or solving problems. It's all very man-with-beard-in-his-shed type stuff, and I think we could do with more crazy men with beards building things in their sheds. I'd like to be one myself - but I'm more of a man-with-stubble-working-in-the-spare-room. But I'm doing the best I can, okay.

Iced latte saved my life

Well, two iced lattes actually. Yesterday, I was presenting at an implementation workshop in Westminster. It was the hottest day for 7 years apparently. It was over 32°C in London. Just riding on the tube was enough to make you sweat. It really gets hot down there.

So when I got near the venue, I went looking for some kind of cold coffee type beverage. I ended up just round the corner, in Victoria ... where I found a Coffee Republic.

[Photo taken from google street view]

So I went in and bought an iced latte - and didn't regret it. Then, in the afternoon, when my presentation was over (which was in a non air-conditioned room) and I was baking hot, I went back to Coffee Republic for another one.

I then journeyed back down into the tube, with my iced latte and the ice cubes still rattling round inside the cup. There were many envious (even desperate) glances from other passengers as I sipped on my ice cold coffee. Sorry guys.

So, if you're in the Westminster/Victoria area of London, try the Coffee Republic down there. The coffee was great and the service came with a smile too. I expect I'll be going back next time...

Self configuring embedded database in C#

I recently needed to write a C# desktop application which used its own small database. I wanted something that could be easily deployed with xcopy, so I didn’t want to connect to any external database engine. So I used SQL Compact Edition v4. Since I’ve recently been having success with PetaPoco as a micro-ORM type thing, I thought that I’d try that too. I had the idea that the database could configure itself when the program starts, so that’s what I set out to try. I was surprised how easy it was. This is what I did to make a proof-of-concept:

  • Create an empty Console Application in Visual Studio 2012
  • Use NuGet to add references to PetaPoco.Core (v5.0.1) and Microsoft.SqlServer.Compact (v 4.0.8876.1)
  • Have this code as the C# program

That’s all there is to it, easy. If you corrupt the database, then the code will re-create a clean database file next time it runs. You don’t need to include an sdf file in your project, because the database will be created by the code. Obviously this is an extremely trivial example, but it seems to work great.

Creating self-signed test certificates with Bouncy Castle

I've been messing about with Bouncy Castle as a means to create self-signed X509 certificates for testing purposes. Essentially I've taken the information from these three blog entries here and here and here, making them into one piece of code. It seems to be a pretty nifty way to make test certificates using C#.

Here are some instructions: first, create a new Console Application in Visual Studio (I'm doing .Net 4) and use NuGet to add Bouncy Castle (I've got v1.7.0) to the project. Then simply use the attached code example. It demonstrates creating a fictional type of security certificate and saving it to disk.

Obviously, I can't take the credit. The clever stuff is all done by Bouncy Castle, and Roger's Blog has demonstrated how to write the C# code. All that I've done is put it together in one place.

The result is a .net X509Certificate2 object which can be used in the normal way.

Random inspiration from Accelerated Massive Parallelism

For some reason I've started looking at the sample code for Microsoft's C++ AMP: Accelerated Massive Parallelism. The temptation to offload work onto the GPU has finally proved too much. Actually, I've looked at various ways to do that in the past, but then slowly walked away...

This time I got as far as calculating the Mandelbrot set by means of C++ AMP, but I won't bore you with that, and besides, Microsoft have provided their own sample code and it's probably better than my version. But when I looked that their code for calling C++ AMP from C# code I was reminded how simple it is to call C++ code from .Net, and I started to wonder why I don't do this more often - even without any Accelerated Massive Parallelism trickery. For example, say you're doing some repetitive calculations, why not do the maths in a small bit of C++ code and get a huge speed improvement? It's not that difficult to do a simple calculation in C++ and you don't need lots more code.

So I spent a lunchbreak writing a sample project to calculate the biggest distance found between an array of points (x,y coordinates). I implemented the same calculation in both C# and C++ and then ran it on an array of 10,000 random points. Using the C++ code, without any help from AMP, it was about ten times faster - even thought I was calling it from within my C# console app:

Wow. That's quite a big difference. Note: I'm computing the distance between every combination of two points in the array and returning the biggest distance found.

Now ... I may not be able to do this type of stuff during my day job, where everything needs to be done in C# only, but I think that I may be tempted to do this type of stuff in my own time when I'm just messing around.

Testing for memory leaks in C

Whilst wandering round the Cambridge University bookshop the other weekend, I noticed a book called Memory as a Programming Concept in C and C++. It was just too tempting, so I bought a copy.

Nobody has ever taught me about memory management in C, I've worked this stuff out from experience and probably by reading other peoples code. So it's nice to read a tutorial - even if I've already been doing this stuff for years. If nothing else, it's made me think more carefully before I dynamically allocate some memory. And I thought that I was pretty careful in the first place.

So I decided to write some debug code to test for memory leaks in my H2D2 virtual machine. I was pretty confident I would not find any problems. But, I did actually find a calloc() where I had meant to put a realloc(). A silly mistake where I wasn't being careful enough. This was leaking memory, because I had a malloc() which was being followed by a calloc() instead of realloc() - so the memory allocated in the first place was not getting freed up (only the memory from the calloc() was getting freed up).

Here is an example of the debug code that I've written: view C code (as a text file).

You just need to replace your calls to malloc, calloc, realloc, and free with rmalloc, rcalloc, rrealloc, and rfree. As you can see from the code, it simply maintains an array of pointers to dynamically allocated memory, so when your code exits you can see if there are any 'dangling' pointers left over. Simple, but effective. If there are no pointers left when your code finishes then you're probably not leaking memory. Nice.

Anyway, I'd be happy to recommend the book 'Memory as a Programming Concept in C and C++', I've found it a useful recap on many things and a good reminder to be extra careful with dynamic memory. It's also been fun to write some memory debugging code, which is a tangent that I may not have gone down if I had not bought the book. Although this book was published 9 years ago, it's nice to see a good book on C programming that's less than a decade old. Thanks Frantisek!

Some H2D2 progress, whilst messing with Run Length Encoding

So progress on my programming language, still codenamed H2D2 continues on background threads :-) I've not given up, but I haven't made many posts on the blog about it recently. It was probably getting boring anyway, and recent work has been tidying things up and writing unit tests - which does not make for interesting reading.

One thing that I wanted to try, was to compress the H2D2 bytecode so that it doesn't take as much time to serialise a running program between machines. I believe that Java JAR files are zipped, so I thought that I'd do something similar with H2D2. But I also wanted something that I could write myself that didn't take an age. Since I'm doing this for fun, it's nice to start with a blank page and go from there. So anyway, my web wanderings brought me to here. That page talks about Run Length Encoding - something that I implemented once myself back in the 1980s (or maybe the early 90s) - not that I have the source code anymore. But I read about the RLE PackBits algorithm and thought it might be a good way to go. Not a massive processing overhead and reasonably simple to implement. I chose not to look at the example souce code provided ... and just to write it from scratch myself. It sounded like fun.

It took me about an hour whilst drinking my morning coffee to write the compression code, and an hour or so in the evening to decompress the data back to the original values (I also wrote some unit tests to check for any obvious problems I could see). It seems to work OK: when I compress H2D2 bytecode I'm getting about a 30% reduction. It's not as good as zip compression obviously, but it will do for now.

I'm sure my code won't win any prizes for optimisation or style, but at least it works. I guess it might also come in handy if you were trying to squash some data on a microprocessor without needing a massive library or lots of processing power. So here is an example C program:

// Example RLE compression with PackBits
// ...just an example, not production code :-)

#include <stdio.h>
#include <string.h> // for memset

void fgrow(char*, char*);
void fshrink(char*, char*);

int main(int argc, char *argv[])
{
  if (argc==4 && strlen(argv[1])==2 && argv[1][0]=='/')
  {
    switch (argv[1][1])
    {
      case 'c':
        printf("Compressing... ");
        fshrink(argv[2], argv[3]);
        printf("done!\r\n");
        return 0;

      case 'u':
        printf("Uncompressing... ");
        fgrow(argv[2], argv[3]);
        printf("done!\r\n");
        return 0;
    }
  }
  
  printf("Usage: RLE { /c | /u }  \r\n");
  printf("   use the /c switch to compress or the");
  printf(" /u switch to uncompress\r\n");
  return 1;
}

void fgrow(char* inpath, char* outpath)
{
  char ch=0, hdr=0;
  FILE* op = fopen(outpath, "wb");
  FILE* mp = fopen(inpath, "rb");

  while (!feof(mp))
  {
    hdr=fgetc(mp);
    if (feof(mp)) break;

    if (hdr>=0) // copy bytes verbatim
    {
      for (; hdr>=0; hdr--)
      {
        ch = fgetc(mp);
        fputc(ch, op);
      }
    }
    else // copy a run of bytes
    {
      ch=fgetc(mp);
      for (; hdr<=0; hdr++)
      {
        fputc(ch, op);
      }
    }
  }
  fclose(mp);
  fclose(op);
}

void fshrink(char* inpath, char* outpath)
{
  char buf[128];
  memset(buf, 0, 128);
  int n=0, hdr=0, runlength=0;
  FILE* op = fopen(outpath, "wb");
  FILE *mp = fopen(inpath, "rb");

  while (!feof(mp))
  {
    buf[n++] = fgetc(mp);
    if (feof(mp))
    {
      n--;
      break;
    }

    if (n==128)
    {
      // buffer is full
      hdr=127;
      fputc(hdr, op);
      fwrite(buf, 1, 128, op);
      memset(buf, 0, 128);
      n=0;
      continue;
    }

    if (n >= 3 && buf[n-1]==buf[n-2] && buf[n-2]==buf[n-3])
    {
      // last three bytes are the same
      hdr = n-4;
      char symbol = buf[n-1];
      char tmp = symbol;
      if (n > 3)
      {
        fputc(hdr, op);
        fwrite(buf, 1, n-3, op);
      }
      runlength=3;

      while(!feof(mp) && tmp==symbol)
      {
        tmp = fgetc(mp);
        n++;
        runlength++;
      }
      hdr = 2-runlength;
      fputc(hdr, op);
      fputc(symbol, op);

      // put the different byte back for next read
      if (!feof(mp)) ungetc(tmp, mp);
      memset(buf, 0, 128); 
      n=0;
    }
  }

  // flush the buffer if not empty...
  if (n > 0)
  {
    hdr = n-1;
    fputc(hdr, op);
    fwrite(buf, 1, n, op);
  }

  fclose(mp);
  fclose(op);
}

Obviously this compression will only shrink files which have runs of the same repeated byte value in them. Otherwise it may increase the file size. You can try it from the command line and pass /c as the first argument to compress a file and then use /u to decompress the file back. The next two arguments need to be the names of the input and output files respectively. This example doesn't have any error trapping, so if the file fails to open, or something like that it will go bang. But that would not be hard to add.

Of course, in my H2D2 code, I'm using the same algorithm, but I've adapted the code to work on blocks of memory rather than on files. But in H2D2 I have made a VFILE struct to work alongside the C FILE pointers and I've added some functions that work the same as the C IO functions but taking my VFILE pointer instead. This means that I can easily swap code to operate either on disk or in memory...