Docker and the MEAN stack

I have been reading the book: Write Modern Web Apps with the MEAN Stack: Mongo, Express, AngularJS, and Node.js and wanted a way to mess about with some examples on the various machines I use (since I'm frequently switching between Mac, Windows and Linux boxes). So I decided to try and Dockerify the process a bit.

I wanted to build a Docker container that I could use on whatever platform I happened to be sitting in front of. I also wanted to be able to run Node and Mongo in a single container. I realise that outside the development environment you'd likely want to split them, but for some simple development I didn't want to have to start up multiple containers.

Of course, when the container runs, I would want to check that it is working, so I also set about including a very simple MEAN stack application as a kind of "Hello, World!" example. But by using Docker Volumes it is easy to replace this default app with something on your local hard disk or perhaps with a Docker Data Volume Container which only needs to contain your app. So the simple default app just needs to prove that Node and Mongo are running.

So my first question was how to go about running both Node.js and MongoDB in a single container, and the solution I have gone with is documented here. It adds a docker-friendly init system to the Ubuntu image. This means that when the container is shut down the init process will try to shut down other processes gracefully, rather than leaving them to be forcefully killed by the kernel. The init process will also reap any zombie processes which may accumulate.

Having a suitable base image, I could then build my own container having specific versions of Node.js and MongoDB. Using specified versions means that I will have a stable set of binaries which will give repeatable behaviour during my development and testing. I don't want to waste time with version conflicts.

Anyway, the result can be found here: https://github.com/davidsblog/node-mongo/blob/master/Dockerfile. I am still tinkering with it, but if you look at the Dockerfile you'll probably get the idea. Both the location of the Node app and the data directory for MongoDB are set up as volumes, so you can override them at runtime. The container then just becomes an engine based on a set of stable binaries, and you can just provide it with your own code and data. You can test it like this:

docker run --rm -p 8080:8888 davidsblog/node-mongo

...and then point your browser to port 8080 of your container. You should see a simple app which allows you to post comments, where the comments are saved to MongoDB. But to switch out the default app to some of your own code you can do this (assuming you are in the directory where your server.js is located):

docker run --rm -v ${PWD}:/vol/node/start -p 8080:8888 davidsblog/node-mongo

...and you should now be running your own program (as long as it is called server.js and assuming it works on the same port).

In the container, the Node.js code is run via Nodemon, which means that when you make changes, Nodemon will restart the app for you - no manual restarting.

In the meantime, I can now mess about developing applications using the MEAN stack without installing everything on my local machine. Cool!

Easy Windows Service Development

I've just discovered Topshelf for creating Windows Services in .Net. It makes building Windows Services a breeze, and you can also run them in the Visual Studio debugger easily too. Just look at the simple code example here ... and you only need to create a console app to make this example work.
Adding Topshelf to your project is easy thanks to NuGet. Your service code just needs to be a plain object and doesn't need to inherit from any special base class. It makes the whole thing extremely simple.

When you've compiled your console app, you can run it in the debugger to check it, and when you want to install it as a proper service, you just pass "install" as a command line argument when you run the exe. To remove the service you re-run the console app and pass "uninstall" on the command line.

MODE_W - a minimal desktop in javascript

I decided that since I've been writing my new H2D2 programming language, it would be good to have some kind of web-based environment to run some H2D2 programs in. So I started writing something in javascript to try and do that. I started calling it MODE (Minimal Operation Desktop Environment) but then I decided to add 'W' for web as well, so I'm naming it mode_w for the moment.

The main idea was for it to be minimalistic - so it's a character based system only. I always remember thinking that the work Microsoft did with MinWin was cool, so that's one of my influences. I always remember that Windows flag logo animated from ASCII characters...

Since I wanted to be able to run several things at the same time, I built a primitive window based environment - but one that works with keyboard control only, and in a browser.

Already I've got something which can move, resize, open and close windows. Everything is simply rendered into a <pre> tag with a monospace font, so theres no mucking about with <div> blocks or anything, just an array of strings that represent the screen - one string for each line on the screen.

Each 'window' draws its own content (which scrolls when it runs out of space) and this means that I can do different things in different windows. So each window becomes a kind of widget, in a text-only kind of way.

I have also allowed character input into each window which can post back a command to a webservice, and this means you can have a command processor running as a web service somewhere else. Of course, different windows could point to different command processors, so a variety of tasks can be done from the one environment.

Here's a screenshot of how it looks so far:

So before I get H2D2 running in mode_w I'm building a few experimental widgets, like the google search widget, just to see how well it works and to iron out any issues. The google search widget allows you to type in a search term and see the top matching URLs.

And maybe I'll build an expression parsing widget which could also be used as a calculator. But after that I will try and get it to host some H2D2 programs, since that is why I made it in the first place...

Implementing Function Calls

I've been busy lately adding the concept of function calls to my H2D2 programming language. It hasn't been as hard as I thought, but there has been a reasonable amount of code to write. I have added the notion of scope as well, so that when a function returns, any local variables go out of scope and the H2D2 virtual machine can free the memory allocated to them. Currently, functions can return a value (although it's possible in H2D2 to return nothing from a function too) ... but I can't yet pass parameters to functions, that's on my TODO list, I'll work on that next.

So here is the current test program that I'm using to see if it all works:

test()
loop(s=48)
  print 10
  print 13
  print s
  print 32
  x=test()
repeat (s=s+1 if s<58)
print x
return

function test()
  loop (s=65)
    print s
  repeat (s=s+1 if s<91)
return 42

This program simply outputs some ASCII characters, the result comes out like this:

ABCDEFGHIJKLMNOPQRSTUVWXYZ
0 ABCDEFGHIJKLMNOPQRSTUVWXYZ
1 ABCDEFGHIJKLMNOPQRSTUVWXYZ
2 ABCDEFGHIJKLMNOPQRSTUVWXYZ
3 ABCDEFGHIJKLMNOPQRSTUVWXYZ
4 ABCDEFGHIJKLMNOPQRSTUVWXYZ
5 ABCDEFGHIJKLMNOPQRSTUVWXYZ
6 ABCDEFGHIJKLMNOPQRSTUVWXYZ
7 ABCDEFGHIJKLMNOPQRSTUVWXYZ
8 ABCDEFGHIJKLMNOPQRSTUVWXYZ
9 ABCDEFGHIJKLMNOPQRSTUVWXYZ*

Simple stuff, but it demonstrates calling the "test()" function like a statement (ignoring the return value) and also assigns its return value to a variable too. I have also done some work allowing this type of code to be written out as labelled bracket notation. Essentially I'm treating each function like a seperate program, starting with its own root node, and then each call to that function simply points to the correct root node. This is how the above program looks if I render the labelled bracket notation generated from the bytecode:

Currently, when I parse the source code, I simply scan for function definitions and emit the code for them and then go back to a 'bootstrap' node which is the programs entry point. The starting point of the source code is pointed to from the 'bootstrap' node. It seems to work fine, and it means that programs always execute from node zero (which I'm making sure is always the bootstrap node).

In other news, I compiled H2D2 as a Windows dll, and used Interoperability to call it from C#, this worked a treat and was extremely fast. With the dll version, any output that the H2D2 program creates is returned as a pointer to a string, so after the program has been run the C# code needs to make an additional call the dll and tell it to free the string containing the output. But, my mandelbrot test program runs in about 200ms this way, since it doesn't have to muck about with writing to the screen. Once the C# program has all the output it can write it in one go rather than a character at a time, which is much quicker.

Now that I have installed Visual Studio 2012 I might be able to hide H2D2 behind a websocket interface. This should allow for a 'chatty' interface to H2D2 where you can ask it to run programs and get the results returned. This is also on my TODO list :-)

Running H2D2 cross platform

So one of my goals in developing DALIS/H2D2 was to make it possible to run a single instance of a program on multiple platforms. Since the H2D2 virtual machine can run code for a timeslice and then persist the entire state of the running program, it should be possible to put an executing H2D2 program into hibernation, move it to a different platform (ie a machine with an entirely different type of OS or processor) and then carry on running the same program. No matter what point the program was frozen at, the code should be able to carry on where it left off.

Well I've now gotten to the point where I can test that theory. The first experiment is to run a program on Windows for a few milliseconds and then complete the execution on Linux. To make this work I needed to make sure that all my data was persisted in sizes that are the same on different platforms, so I need to use int32_t instead of int, that type of thing. Since I've written my code in a cross-platform way and since I'm using data sizes that will be the same on different platforms everything should just work. So here we go, I'm running my mandelbrot program on Windows for 200ms:

...so that outputs a file called 'demo.hby' which is the H2D2 bytecode including its persisted state (all the program instructions, the call and data stacks and the values of all variables). Now I need to move that file to my Linux box and run the code from where it stopped. On the Linux machine I have already compiled the H2D2 virtual machine from source using GCC of course. Here goes:

Awesome! It works! I guess it's not much more than a neat trick at the moment, but I think it's an achievement of sorts. If you had some kind of long running process, it might be handy to be able to wake it, run it on whatever machine was available, and then put it back into hibernation. Okay, you can't start re-writing all your business logic in H2D2 just yet... but it's early days. This is why I always imagined DALIS / H2D2 to be a cloud based language, where you don't care what type of platform or processor is being used from one moment to the next.

So the next obvious experiment is to do the same thing, but on the Raspberry Pi... maybe I'll do it in reverse, by starting the program on the Raspberry Pi and then finishing it on Windows.

Z88 Serial Communication - part 2

I have been meaning to make a blog entry about this for ages, but haven't gotten round to it yet.  This is part 2 of my work on establishing serial communications with the Cambridge Z88.  In the first part I described how to install a small Basic program on the Z88 ... this part completes the process by running a matching program on Windows.  This allows data to be sent back and forth, but only a single file at a time.  It uses my own protocol, and allows binary files (like Basic programs) to be sent.

Z88 Serial Communication for Windows

This is program needs .Net Framework v2 (click here for the for the x86 version), so that needs to be installed first, but you've probably got it already.  I've only tried it on Windows 7, although I don't see why it shouldn't work on Windows XP or Vista.

If you have installed the Basic program on your Z88, you can now use this program on Windows.  Just copy the files to a folder and run the .exe file.

NOTE: anything above 9600 baud might work; but I have not found it reliable.  This program works with the genuine Z88 transfer cable, since it uses hardware handshaking to prevent the buffer from filling up.  The serial settings on the Z88 will be set up using the normal Panel settings.  I’ve been using these settings - Transmit baud: 9600, Receive baud: 9600, Parity: None and Xon/Xoff: No.

Sending from the PC
Run the Basic program on the Z88.  Press ‘R’ and <Enter> to say that you’d like to receive a file.  Then enter the name of the file, the name that will be used on the Z88 and press <Enter>.  The Z88 will wait for the file to arrive.  Now on the Windows program select ‘Send’ along with the COM port settings you’re using.  Click the ‘...’ button to browse for the file on your PC.  Finally, when you have selected the file click ‘Transfer’ and you should see the file being transmitted.

Sending from the Z88
In the Windows program select ‘Receive’ along with the COM port settings.  Click the ‘...’ button and select the file name and the folder where the file should be saved.  Then click the ‘Transfer’ button.  The PC will wait for the file to arrive.  Now, run the Basic program on the Z88.  Press ‘S’ and <Enter> to say that you’d like to send a file.  Then enter the name of the file on the Z88 that needs to be transmitted and press <Enter>.  The Z88 will now transmit the file to the PC.