Putting NETMF on a diet – Part 2

January 4, 2012 1 comment

In the previous post I complained about my .Net Micro Framework application not fitting into the 100k of space available on the Fez Panda II. I optimized it by inspecting the code and got a 160% improvement.

That was all done on the PC by looking at the PE files in the bin directory.

Today I will look at the effect the optimizations had on the actual hardware. I’ll be deploying the solution and tracking the statistics.

To recap, the sizes of the .pe files in the bin directory are currently standing at these numbers:

Class kb
Realiser.NetMF.DeviceDrivers.GraphicLcd 5 344
Realiser.NetMF.DeviceDrivers.Keypad5x5 2 324
Realiser.NetMF.DeviceDrivers.CharLcd 6 204
DataSpot.Firmware 16 792
Realiser.NetMF.CSDB 4 720
Realiser.NetMF.DeviceDrivers.Display 4 496
Realiser.NetMF.DeviceDrivers.Rfid 7 120
  47 000

 

I then tried to deploy to the device… Error!

At first it is hard to spot what is going on because Visual Studio simply gives this error:

image

When this first happened I checked cables, firmware versions and who knows what else… After some digging I found this message obscured in the Output window, under the Micro Framework Device Deployment dropdown:

image

That is a very poor way of telling me that my app is still too big… 102,752 bytes.

According to the Fez Panda  II specs, I should have “lots of FLASH and RAM” – 148kb Flash and 62kb Kb RAM to be exact.

I split the program memory into two sections – one for a custom bootloader (48kb) and one for the main app. This should leave me with 100kb – or 102,400 bytes. From the looks of this I am currently 352 bytes over!

Let’s see if we can squeeze that down.

I spent a lot of effort last time to optimize the code – so there’s not much more I think I can do there. I know there are a lot of Debug.Print statements in the solution, so the easiest is just to get rid of these for now…

Find Debug.Print and replace with //Debug.Print (so I can put them back later!).

36 Matches found – but alas, no luck. 101,556 bytes now. I now also know that 100kb does not mean 102,400 bytes here… I’ll have to keep shaving off to find the magic number.

In my Rfid module there are a bunch of methods created per the datasheet for the device I’m using, but I haven’t implemented them yet. They are “empty” methods like this:

public Response WriteBlock(TransponderType transponderType, int blockAddress, string data)
{
    throw new NotImplementedException();
}

This is a good time to also understand the relationship between the pe file and the actual size on the device.

Before taking out an unused method, the size of the pe file is 7,048 bytes and the image is currently 101,556 bytes.

After commenting it out, the pe file is 7004 bytes and the image 101,512 bytes.

So, based on this experiment, the size of this method is 44 bytes in the pe file and 44 bytes in the image. This is good news! It means there is a direct correlation between the pe file size and what goes on the device.

I commented out another one:

public Response ReadPasswordedBlock(TransponderType transponderType, int blockAddress)
{
    throw new NotImplementedException();
}

– 6,964 and 101,472 – so 40 and 40.

Where is the extra 4 bytes? Looks like it is because of the 3rd parameter on the first method vs. only 2 on the second.

Another one… This time to see what effect the body of the method has:

public Response WritePasswordedBlock(TransponderType transponderType, string password, string data)
{
    throw new NotImplementedException();
}

First I commented out the whole thing… 6,916 and 101,424 – so 48 bytes (and this one also has 3 parameters).

Now I commented out just the body… and got an exception. It needs to return something. I replaced it with “return null”. This is kind of breaking my experiment, but let’s see anyway… 6,968 and 101,476… 52 bytes. Looks like “return null” is 4 bytes heavier than Throw new NotImplementedException!

My conclusion is that a typical method, with a return object is about 28 bytes plus 4 bytes per parameter plus 4 bytes for “an empty body”. I’m sure this should be documented somewhere – and finding that document is a mission for next time🙂

Ok, back to the original problem…

After eliminating a lot of code, I got it down to 99,504 bytes – and still no luck.

I knew I wasn’t using the GraphicLcd assembly yet, so I removed the reference from the Firmware project. Success! At 94,160 bytes it deployed.

I don’t have the exact numbers yet, but I’m going to venture a guess that the size of the deployment must be less than 96kb – or 98,304 bytes.

Maybe this is an opportunity to see the correlation between the size of a pe file on disk and what goes to the device…

With GraphicLcd referenced, it says the size is 99504. Without it, 94,160. pe file size on disk is 5344 – exactly the same as what is deployed. So I think one can safely assume the size of the pe files is the same as what will be used in Flash on the device.

The sum of all the pe files in my solution is:

Class kb
DataSpot.Firmware 16 468
GHIElectronics.NETMF.USBClient 8 640
Realiser.NetMF.DeviceDrivers.Rfid 6 576
Realiser.NetMF.DeviceDrivers.GraphicLcd 5 344
Realiser.NetMF.DeviceDrivers.CharLcd 5 116
Realiser.NetMF.DeviceDrivers.Display 4 420
Realiser.NetMF.CSDB 3 616
GHIElectronics.NETMF.System 2 496
Realiser.NetMF.DeviceDrivers.Keypad5x5 2 212
FEZPanda_II_GHIElectronics.NETMF.FEZ 904
GHIElectronics.NETMF.IO 664
  56 456

This doesn’t fit.

Without GraphicLcd, this fits:

Class kb
DataSpot.Firmware 16 468
GHIElectronics.NETMF.USBClient 8 640
Realiser.NetMF.DeviceDrivers.Rfid 6 576
Realiser.NetMF.DeviceDrivers.CharLcd 5 116
Realiser.NetMF.DeviceDrivers.Display 4 420
Realiser.NetMF.CSDB 3 616
GHIElectronics.NETMF.System 2 496
Realiser.NetMF.DeviceDrivers.Keypad5x5 2 212
FEZPanda_II_GHIElectronics.NETMF.FEZ 904
GHIElectronics.NETMF.IO 664
  51 112

So, off to find the elusive 50k robbing my space…

 

When you deploy a NETMF app, the device gives you back some statistics about the files.

Apologies for the long section that follows… Just scroll over it. I will refer back to this later in the article…

 

Create TS.

Loading start at 4ebac, end 56d84

Attaching file.

Assembly: mscorlib (4.1.2821.0) (3880 RAM – 33236 ROM – 19134 METADATA)

   AssemblyRef    =        0 bytes (       0 elements)

   TypeRef        =        0 bytes (       0 elements)

   FieldRef       =        0 bytes (       0 elements)

   MethodRef      =        0 bytes (       0 elements)

   TypeDef        =     1112 bytes (     139 elements)

   FieldDef       =      272 bytes (     135 elements)

   MethodDef      =     1572 bytes (     786 elements)

   Attributes      =        0 bytes (       0 elements)

   TypeSpec        =       16 bytes (       4 elements)

   Resources       =      232 bytes (      29 elements)

   Resources Files =       16 bytes (       2 elements)

   Resources Data  =      437 bytes

   Strings         =     1551 bytes

   Signatures      =     2126 bytes

   ByteCode        =    11702 bytes

Loading Deployment Assemblies.

Attaching deployed file.

Assembly: DataSpot.Firmware (1.0.4386.18398) (1552 RAM – 16468 ROM – 5217 METADATA)

   AssemblyRef    =       52 bytes (      13 elements)

   TypeRef        =      220 bytes (      55 elements)

   FieldRef       =        8 bytes (       2 elements)

   MethodRef      =      468 bytes (     117 elements)

   TypeDef        =      128 bytes (      16 elements)

   FieldDef       =      144 bytes (      72 elements)

   MethodDef      =      264 bytes (     132 elements)

   Attributes      =        0 bytes (       0 elements)

   TypeSpec        =        4 bytes (       1 elements)

   Resources       =        0 bytes (       0 elements)

   Resources Files =        0 bytes (       0 elements)

   Resources Data  =        0 bytes

   Strings         =     4070 bytes

   Signatures      =      593 bytes

   ByteCode        =     7050 bytes

then lots more for the other assemblies (deleted)…

and finally the totals:

 

Total: (14136 RAM – 127396 ROM – 61195 METADATA)

   AssemblyRef    =      180 bytes (      45 elements)

   TypeRef        =     1252 bytes (     313 elements)

   FieldRef       =      148 bytes (      37 elements)

   MethodRef      =     1864 bytes (     466 elements)

   TypeDef        =     3064 bytes (     383 elements)

   FieldDef       =     1364 bytes (     675 elements)

   MethodDef      =     3952 bytes (    1969 elements)

   DebuggingInfo  =     1996 bytes

   Attributes      =       48 bytes (       6 elements)

   TypeSpec        =       28 bytes (       7 elements)

   Resources Files =       72 bytes (       3 elements)

   Resources       =      304 bytes (      38 elements)

   Resources Data  =     1184 bytes

   Strings         =    18978 bytes

   Signatures      =     7955 bytes

   ByteCode        =    43475 bytes

The debugging target runtime is loading the application assemblies and starting execution.
Ready.

‘Microsoft.SPOT.Debugger.CorDebug.dll’ (Managed): Loaded ‘C:\Program Files (x86)\Microsoft .NET Micro Framework\v4.1\Assemblies\le\mscorlib.dll’, Symbols loaded.

and some more load messages (deleted).

and then the GC message (which I’ll analyze in the future):

GC: 1msec 27156 bytes used, 37224 bytes available
Type 0F (STRING              ):     96 bytes
Type 11 (CLASS               ):    720 bytes
Type 12 (VALUETYPE           ):    132 bytes
Type 13 (SZARRAY             ):   1080 bytes
Type 15 (FREEBLOCK           ):  37224 bytes
Type 16 (CACHEDBLOCK         ):     72 bytes
Type 17 (ASSEMBLY            ):  16224 bytes
Type 18 (WEAKCLASS           ):     48 bytes
Type 1B (DELEGATE_HEAD       ):    108 bytes
Type 1D (OBJECT_TO_EVENT     ):     48 bytes
Type 1E (BINARY_BLOB_HEAD    ):   5940 bytes
Type 1F (THREAD              ):    384 bytes
Type 20 (SUBTHREAD           ):     48 bytes
Type 21 (STACK_FRAME         ):    612 bytes
Type 27 (FINALIZER_HEAD      ):     48 bytes
Type 31 (IO_PORT             ):     72 bytes
Type 34 (APPDOMAIN_HEAD      ):     72 bytes
Type 36 (APPDOMAIN_ASSEMBLY  ):   1452 bytes

 

Is there a correlation between the pe file size and size of the assembly in ROM? We already know that what’s in the pe file goes to the device as-is, but where does it end up?

I certainly was not going to tally up the numbers by hand, so I developed a quick Excel spreadsheet with some macros to parse the Debug output.

 

The link is here: https://rudieshepherd.files.wordpress.com/2012/01/uprofiler.xlsx

Be warned… It will complain that the file is corrupted. Right click the link and Save As uprofiler.xlsm because it contains macros and WordPress don’t allow .xlsm files. You also need to enable macros before it will do anything. Contact me if you need help getting this to work.

 

Here are the results:

image

and a nice graph to go with it…

image

So it is clear that the heavy hitters are mscorlib, System.IO and my own DataSpot.Firmware.

I checked the numbers against the size of the pe files and ROM is an exact match – so it is safe to look at the pe files as an indication of what will occupy ROM in the end. I still cannot say for sure if these IL files actually go on the device in the end… I’m pretty sure there’s a conversion to HEX files in the process somewhere as well.

So where does 94,160 come from as reported in the Micro Framework Device Deployment window before? Turns out that it is the size of all the assemblies minus the size of mscorlib – so 127,396 – 33,236 = 94,160.

Can I find the “missing” 50k now?

Sure… It’s the Microsoft.SPOT assemblies. This makes sense because those are already present on the device in the NETMF firmware and don’t need to be deployed.

Here are the numbers without the MS assemblies:

image

Same numbers as the sum of the PE files before.

and the MS assemblies on their own (without mscorlib):

image

So beware… When they advertise “148kb for user application” it really is a lot less than that if you use anything in the MS core assemblies and even less if you use a custom bootloader…

Roughly only about 50k for your own stuff.

That is sooooo unfair… I started with a device with 512kb flash and ended up getting 50k for my own use. Sure, the framework saved me tons of code I’d have to write myself, but because NETMF deploys everything you might need as opposed to only the things you actually use, you get a lot of bloat thrown into a small package.

<rant>

I have a big problem with this… NETMF is targeted at “micro” devices. This is not micro anymore. Everything is pushing towards bigger and faster processors – and in the process we just accept the bloat. Soon you’ll need a 200 MHz+ processor with megs of ROM and RAM to run something that we’ve always been able to run on a AVR-type device in 1k of ROM and a few bytes of RAM. I am tempted to port my app back to AVR – and having a tough time defending why I should still use NETMF for something so basic.

I think NETMF is going in the wrong direction… It should become leaner, like a “managed microcontroller” – with C#, not bloated like an operating system. We already have bloatware like Embedded Linux and Win CE for that segment.

</endofrant>

 

All this and I still haven’t solved my problem… I need another 5k – 10k so I can fit my GraphicLcd library as well as some extra functionality I still need to add.

One more table and graph from the spreadsheet shows me this:

image

It’s a bit hard to see here, but what it is telling me is:

– mscorlib is a hog. It has tons of bytecode, signatures, methods and types. Can’t do anything about it though.

– DataSpot.Firmware has a lot of Strings (4070 bytes)

– GHI’s USBClient also has a lot of strings – I’m assuming for USB device descriptors and such?

– Rfid can also do with a cleanup (1710 bytes of Strings)

– System.IO uses a big chunk of bytecode (almost 7k) – can I do without it?

 

Looking at the Strings, there isn’t much I can get rid of. There is some overlap between the strings used in the Keypad mapping and the handler in the main program. What will happen if I move these to a resource file?

So, the baseline for Firmware is 16,468 bytes at the moment.

I replace a string like “Power” with an empty one…

Result: 16,460 bytes – so 5 characters = 8 bytes – probably 1 byte per character + 3 bytes overhead.

I removed the Resources file in the previous article – so now I’m putting it back. That is a hit of 344 bytes off the bat with nothing defined yet.

The resource definition for 24 strings (70 characters) takes 736 bytes. Not looking good so far.

This got me down a rabbit hole… Changing the strings to a resource enumerator had implications all over the code. It might be more efficient to use the enumerator than strings, but I find using strings to be easier to debug plus I do some funky stuff with manipulating the strings in the keypad class – so I’d rather leave it as it is.

There were some places I used long strings… At first I implemented a class where I want to override certain methods in classes that derive from it. I had a definition like this:

public virtual void UpdateDisplay()
{
    throw new NotImplementedException("Implement UpdateDisplay in the device-specific Navigator class.");
}

But that is not very clever… This is what abstract is for… so I changed it…

public abstract void UpdateDisplay();

Don’t know what I was thinking before… Late night coding I suppose!

Result: 16,164 bytes – so a saving of only 304 bytes – but the code is much cleaner now.

I’m seriously running out of options now… So I try some of the desktop tools that are part of the .Net Framework.

ILDASM is the IL Disassembler described here: http://msdn.microsoft.com/en-us/library/f7dy01k1.aspx

This doesn’t really tell me any more than I already know, so that’s a dead end (but still interesting). I also used .Net Reflector on the exe file.

My assembly still shows 3777 bytes of strings… That is a small book. Where can they be hiding?

Looking at the PE file again, I see a bunch of readable text like this:

ÿüu  † ÿÿ +z@  –   ÿÿ6 ?   –   ÿÿ6ýb   Æ m A Ó   Realiser.NetMF.DeviceDrivers.CharLcd Realiser.NetMF.DeviceDrivers.Keypad5x5 Realiser.NetMF.CSDB Realiser.NetMF.DeviceDrivers.Rfid GHIElectronics.NETMF.USBClient GHIElectronics.NETMF.IO Realiser.NetMF.DeviceDrivers.Display Microsoft.SPOT.Hardware.SerialPort LcdBase Keypad5x5 CsvRow P1D

These are clearly the names of classes, fields and methods.

One way to reduce the size of this, and gain some security at the same time is to use an obfuscator. I tried the one at http://www.ssware.com/ but couldn’t get it to work with NETMF on the first try. I’ll try again later.

I think that is as much as I can handle for today. I still did not meet my target of getting to under 90kb for the total deployment, but I’ll be sure to try again.

 

This is the point where I make an appeal to Microsoft:

Please please please…

– Finish the native compiler for NETMF that will deploy native code as opposed to bytecode to the device.

– Make the “linker” more clever to strip out unused IL instructions from libraries before deploying them.

– Give us an integrated profiler in the IDE to look at RAM and ROM – and break the stats down to see where things are used.

– If we can’t have an integrated profiler, give us structured Debug output commands that can easily be parsed by VS add-ins so we can write visualizers for them.

Until next time – happy hardware hacking!

Categories: Uncategorized

Putting NETMF on a diet

January 3, 2012 Leave a comment

Isn’t .Net Micro Framework supposed to be “micro” already? Well it is – until you start to write real code for it!

I’m busy developing a handheld data collector device for the agricultural community. This is a fun project to keep my mind occupied with challenges beyond the usual corporate routine.

The project has grown over the past year to the point where all the functionality no longer fits into the memory of a Fez Panda II device I’m using as the base. On this particular NETMF device, I have 148kb of program memory at my disposal. That is kilobytes folks… not megabytes or gigabytes on a PC!

After the last round of changes, I included a self-updating routine to update the program from SD card. This is a very cool feature of GHI’s USBIZI chipset, but unfortunately it comes at a cost of 48kb just for the custom bootloader.

The problem is my original program requires about 110kb of space… and that doesn’t fit into the 100kb I now have. The challenge is thus to put NETMF (or my app actually) on a diet and see if I can fit it in a confined space without losing functionality.

This article will document some of the techniques I used and the things I learnt along the way.

 

The solution structure looks like this:

image

– DataSpot Firmware is the main app.

– CSDB is a CSV file based database;

– CharLcd and GraphicLcd are two classes to deal with different types of displays. They both derive from the Display class.

– Keypad5x5 is a matrix keypad driver.

– Rfid is a driver for the RFID module I’m using to read animal tags.

 

So really not such a big project – so what’s chewing up the bytes? Time for some statistics…

There are some obvious things that can use up resources…

1. Code

2. References

3. Resources

Looking at Code first, my program should translate into byte code that will be deployed to the device. In the \bin\be folder I found these files:

image

.pe Files are the files that will go on the device. pdbx files are debug files. The sizes are not a good indication of the actual size that will be used on the device, but it gives a good relative indication of where the bulk of the code is.

Just looking at pe files, sorted by size, I see this:

image

So, my Firmware (main app) is using the most (20 kb), followed by the USB client and then my CharLcd module. There’s nothing I can do about USBClient (it belongs to GHI), but the rest I can surely trim down.

The first thing I noticed was that I had references in my project do not match the ones in the output bin folder – most notably the Microsoft.SPOT references. Don’t know why… Probably because they already exist in the core framework on the device.

My references look like this:

image

I’ve already trimmed all the references down to only those I actually use, so there isn’t much else I can do on that front.

Next, I looked at the top contenders… Firmware, CharLcd and Rfid. Let’s see if I can put those on a diet.

image

The main Program.cs file is about 500 lines of code.

Navigator is where the magic happens, and that is about 1100 lines of code.

The rest are relatively small.

Lines of code is not a good indicator of size, because the Rfid class is also about 1000 lines, but that is only 7kb. I’ll use a line counter to get and overview anyway…

http://archive.msdn.microsoft.com/LOCCounter

It turns out that the line count in Visual Studio is pretty misleading… Using a “proper” code line counter that ignores comments and blank lines give a much better view of reality. Rfid is big because it is heavily commented🙂

The code in Program.cs is pretty compact already, except for one section where I use a lot of If’s…

void _Keypad_KeyPressed(object sender, string key)
{
    if (Device.CurrentState == Device.StateEnum.LOCKED)
    {
        Device.Sounder.Error();
        return;
    }

    Device.Sounder.Tick();

    if (key == "E3")
    {
        Device.Navigator.TextInputMode = true;
        _KeyBuffer = "";
    }

    if (key == "Lock")
    {
        LockDevice();
    }

etc. (12 statements in total).

Now I wonder what would happen if I coded this as a switch statement instead… More out of curiosity than anything else…

switch (key)
{
    case "E3":
        Device.Navigator.TextInputMode = true;
        _KeyBuffer = "";
        break;

    case "Lock":
        LockDevice();
        break;

    case "Connect":
        Device.Navigator.Alert("Hold down to connect");
        break;

etc.

Result: 20,092 bytes down to 20,068 bytes. So it does help – but very very little.

Next… A lot of Debug statements in there. Theoretically these should go away by themselves if the code is compiled for Release, but I’d like to see. I commented out all the Debug.Print statements with a search/replace. Only 10 occurrences here and down to 19,796 bytes. That is 272 bytes in just the one project. I think it must be long strings… and I do use a lot of those…

Unfortunately most of the strings I  use are different from each other – so it won’t really help to use enumerators or character constants either. I’ll look into that a bit more later.

Next… Unnecessary code.

I originally came across the problem with the bloat when I implemented the System Update function. After learning how it works, I now know that I can do the update in a separate Bootloader application that already takes 48kb. So, next thing to do is to eliminate the system update code from the main application.

1. Remove references to GHIElectronics.NETMF.SystemUpdate.

2. Remove code that uses the referenced assembly. I just build after removing the reference and let the compiler tell me where it is used. 6 Places in this case.

Result: Down to 18,936.

This doesn’t look like much, but another 1kb referenced file for SystemUpdate also disappeared in the process.

My Navigator class is a big one and it also has a lot of large strings – mainly as Exception messages. Just to see the effect, I commented out a bunch of these.

Result: 18,588 bytes – so this is not worth it. I’d rather have good error messages! Undo.

Next… Resources.

There is a Resources.resx file but I know I don’t have any embedded resources, so delete it.

Result: 18,628 bytes. Not bad for just deleting something that’s not used at all.

Next… Get rid of empty classes.

I want to have one source tree for multiple devices. For example, there is a small device that uses a 16×2 character Lcd display and a big device that uses a 128×64 graphics Lcd display. To keep everything tidy, I use compiler directives (#if statements) to bring in the appropriate code and I abstract some of the classes from generic base classes.

One example of such an abstraction is the Lcd16x2Display class.

image

It is part of the PC (PC=Panda/Character if you must know) series device and in this particular case it directly inherits the CharacterDisplay class without adding anything (other than organizing my code around the device):

namespace DataSpot.Firmware.PC_Series.PC162U
{
    public class Lcd16x2Display : CharacterDisplay
    {
    }
}

So, what would happen if I used CharacterDisplay directly and lose the abstraction?

First I excluded the original class from the project and compiled. It told me there are 2 references. I changed these to directly use CharacterDisplay – and put a TODO comment in to change it back if it doesn’t help.

Result: 18,568 bytes – so an overhead of 60 bytes each time you inherit another class. For the sake of readable code and a clean architecture, I’d rather pay this price thank you very much… Undo.

At this point I’m getting rather discouraged. A total saving of 1,464 bytes (7%) from where I started – and there isn’t much more that’s obvious to do.

 

On to the next contender… The CharacterLcd class. We start with 8,232 bytes.

image

Same as before, I eliminate unused references and code.

In this class I have some custom LCD characters and animations.

A custom character looks like this:

using System;
using Microsoft.SPOT;

namespace Realiser.NetMF.DeviceDrivers.CharLcd.Characters
{
    public class SignalStrength0 : CharacterBitmap
    {
        public SignalStrength0()
        {
            byte[] bmp = new byte[8];

            bmp[0] = 0x0;
            bmp[1] = 0x0;
            bmp[2] = 0xA;
            bmp[3] = 0x4;
            bmp[4] = 0xA;
            bmp[5] = 0x0;
            bmp[6] = 0x0;
            bmp[7] = 0x1F;

            this.Pattern = bmp;
        }
    }
}

So this got me thinking… I don’t use any of the characters or animations at the moment – so why are they there then? This is classic YAGNI (You Ain’t Gonna Need It!) at work. I made these when I was playing with the display and wanted to do something fun.

Theoretically these unused classes should not contribute to the overall size of the library, not so? The “linker” should only bring in what is referenced and used – or so I seem to remember from old C days. Surprise! .Net MF compiles and packages the whole thing… whether you use it or not. To prove this, I simply excluded the classes from the project and recompiled.

Result: 6,204 bytes… That is over 2k – and in this case 25% down! Now that is a good return on the investment!

So lesson learnt… Only include code you really need and use in a library. A code coverage tool should be useful here… Maybe a reason to use the Visual Studio Pro instead of Express?

I’ll have a look if including the classes directly in the main project as opposed to a DLL makes a difference too. I’m also not sure at this stage how much actually deploys to the device in the end… That will be a future experiment.

 

The original idea was to have one project to handle multiple device types and compile it using compiler directives. After seeing the issue with unused classes, I am starting to think it is wiser to have one solution file for each device and even specific project files to make sure only what is needed is included. You cannot selectively include or exclude project references based on directives, and once a library is referenced, it is included in its entirety, so I don’t see any other clever way to optimize this aspect than to use separate project files on top of shared source files.

 

Next is the GraphicLcd class. I initially excluded this from the project because it is big and I only needed the CharacterLcd at the time. Simply adding it as a reference to Firmware did nothing to the Firmware file itself, but added 28k to the overall deployment. That is a killer.

image

The library mainly consists of the Canvas class, which does most of the paint work, and a bunch of Font classes.

With the experience of earlier, I now knew this is the place to start looking. I immediately excluded all the Font files from the project. They belong elsewhere as a resource lib anyway.

Result: From 28,332 bytes to 13,372 bytes. Wow.

In this class is also a bunch of UI Controls I developed. Nifty things like a Label and a ProgressBar, Menu and even a Chart control.

YAGNI!

But maybe some day with the right project I WILL need it… In fact I developed them for a parallel project but wanted to maintain one library.

How do I exclude them for this solution but include them in another – without messing around with multiple project files?

The only way I could think of was to use compiler directives.

First I set a compiler directive called INC_UI_CONTROLS in the Firmware (main app) properties:

image

Then I put in compiler directives around each class I wanted to conditionally include:

#if INC_UI_CONTROLS
using System;
using Microsoft.SPOT;
using System.Collections;

namespace Realiser.NetMF.DeviceDrivers.GraphicLcd.UIControls
{
    public class AutoCompleteTextBox : Textbox
    {
        private ArrayList _Items = new ArrayList();
        private int _CurrentSelectedItem = 0;

#endif

The problem with this is it doesn’t work! A directive set in one project does not transfer to another and there isn’t a way to set a “solution wide directive”. If anyone knows a workaround for this I’d appreciate it.

For now I’ll use the Exclude From Project method. This is easy because all the files are already bundled in a folder.

Result: 13,372 bytes to 5,344 bytes.

Lesson learnt: Put things you’d want to conditionally include/exclude in separate folders in the project.

So, from 28,332 bytes to 5,344 bytes… That is significant for a single library. I mean 81% significant!

 

On to the next class… Keypad5x5.

This is a small class at only 3,736 bytes – but it seems big for what it does. Let’s see what we can do.

image

I don’t use IKeypad and GpioKeypad anymore. Those were experiments early on in the development. Exclude them.

Result: 2,324 bytes. Already 1,412 bytes down.

If you recall, I could only shave 1,464 bytes from the entire big 20k Firmware main app… so here I’ve gained the same from a seemingly insignificant library.

Lesson learnt: Don’t overlook the small libraries. Optimize on principles – not size of the library.

The last library is RFID. I’m going to skip this one because I spent a lot of time on it already before publishing it to the TinyClr Code library. Sure, one can always do more, but for now I am satisfied.

What’s the final score?

Class From To Saving Saving %
Realiser.NetMF.DeviceDrivers.GraphicLcd 28 332 5 344 22 988 81%
Realiser.NetMF.DeviceDrivers.Keypad5x5 3 736 2 324 1 412 38%
Realiser.NetMF.DeviceDrivers.CharLcd 8 232 6 204 2 028 25%
DataSpot.Firmware 20 092

18 628

1 464 7%
Realiser.NetMF.CSDB 4 720 4 720 0%
Realiser.NetMF.DeviceDrivers.Display 4 496 4 496 0%
Realiser.NetMF.DeviceDrivers.Rfid 7 120 7 120 0%
  76 728 48 836 27 892 151%

Not bad for an hour’s work!

I happen to notice the “Optimize code” tickbox in the Build properties before. What effect does this have?

Class From To Saving Saving %
Realiser.NetMF.DeviceDrivers.GraphicLcd 28 332 5 344 22 988 81%
Realiser.NetMF.DeviceDrivers.Keypad5x5 3 736 2 324 1 412 38%
Realiser.NetMF.DeviceDrivers.CharLcd 8 232 6 204 2 028 25%
DataSpot.Firmware 20 092

16 792

3 300 16%
Realiser.NetMF.CSDB 4 720 4 720 0%
Realiser.NetMF.DeviceDrivers.Display 4 496 4 496 0%
Realiser.NetMF.DeviceDrivers.Rfid 7 120 7 120 0%
  76 728 47 000 29 728 160%

It only seems to affect the main app – but enough to make a difference, so it should be worth switching on.

In the next instalment I’ll look at how these numbers translate into actual sizes on the device and what else can be done to optimize things.

Until then… happy hardware hacking!

Categories: Uncategorized

Achtung! Birthday boys on the loose!

September 9, 2011 Leave a comment

So it was my son’s birthday and mine shortly thereafter, so I decided to spoil myse… err… him and get some real boys toys… A Panzer III RC tank and some other RC goodies.

This thing shoots Airsoft BB’s! In retrospect not the brightest idea in the hands of 7 year olds – but I got around that problem – more on that later.

The best part of any prezzy… The unboxing🙂

IMAG0192

Everything arrived safe and sound – all the way from Hong Kong.

IMAG0193 

I must admit I was worried about what quality to expect but I was pleasantly surprised. It is made from a heavy plastic and the tracks are nice and sturdy.

Only complaint is the smoke oil leaked out in transit.

The transmitter is 7 channels – drive control, turret left and right, up/down, shoot and then 3 other sound effect buttons.

After a few weeks of playing that transmitter is going to be replaced with a WiFi link anyway when I transform this puppy into a remote viewing, rooten tooten BB shooten attack robot – whahahaha…

Anyway, back to reality…

The boys are ready to take a test drive…

IMAG0197

When I first shot the BB’s, it went right through the target – and it is a rather thick piece of cardboard. This got me worried about the power here. It will definitely hurt bad – or take an eye out. So what to do? There’s no way I can deny the kids the pleasure of shooting at something!

The universe must have been looking out for the kids because one of his other gifts was one of those guns that shoots foam darts. Brainwave! The darts fit perfectly over the barrel of the tank and it shoots very well (without the risk of bodily harm)!

IMAG0201IMAG0200

Here’s a video of the tank in action…

 

And here are some of the other goodies that came in the box – for dad this time🙂

Look carefully… Four motors, four propellers, four speed controls, big chunky LiPo battery… Whatever can be on the cards for the next holidays?

Quadrocopter of course!!!!

  IMAG0194

Categories: Uncategorized

Creating DesignSpark library components the easy way

I’ve recently start trying my hand at DesignSpark after years of being an Eagle fanboy. I must admit, I love Eagle and as a hobbyist it’s given me great service. However, my little hobby project has now progressed to a level where I just need a little more “pro” features, but not enough to justify a pro license – so why not give something new a try?

My project is based on a Fez Panda II controller and a Topway LCD. It also has a membrane keypad and RFID reader.

From this basic layout you can see my problem… It does not fit the 100x80mm limitation in Eagle, so I set about converting it to DesignSpark.

clip_image002

The first thing I wanted to do was to create a library component for the LCD. This doesn’t exist in the standard libraries, so I needed to create it from scratch.

At first I sat with the datasheet and tried to transfer the dimensions to Eagle. I kept on getting it wrong because there are some notations in the datasheet that I don’t understand. What does 2-3.5 and 4-2.2 mean!!!

clip_image004

Anyway, after a lot of frustration it dawned on me that the drawing was already done, to scale, and all I needed to do was to get it from the PDF and into my CAD program.

Enter DesignSpark… and its ability to import DXF format. If I could find a way to get the PDF to DXF, surely I could simply import it?

Lucky for me PDF is a format that translates to DXF very well. I used http://anydwg.com/download/pdf2dxf.exe to do the initial conversion – and hey presto – a working DXF!

After the first import it became clear that the scale wasn’t right. By chance (or good design) DesignSpark has a Scale feature in its import to compensate. Before I could set the scale, I had to determine the scale factor, and here’s how…

First, I created a new empty PCB.

clip_image006

An important setting in the Wizard is the Units and Precision. Because my reference drawing is in mm, I’d like to work in mm on the board too. I set the precision to 4 (after a few attempts) because I found some rounding errors.

I then proceeded to File -> Import the DXF with Scale set to 1.

It looked like this:

clip_image008

Very very close to the original.

To find out the scale factor, I then used the measuring tool to measure between two known points – in this case from Pin 1 to Pin 16.

clip_image010

The spec says this is 38.1mm and my measurement shows 13.3730mm.

To get to the correct scale I need a scale factor of 38.1 / 13.373 = 2.849.

For reasons unbeknownst to me, when I re- imported at this scale, things were off by 0.2345mm:

clip_image012

I can’t explain it, but I simply proceeded to play with the scale factor until I got a perfect fit – in this case it was 2.822.

Once I had the scale right, I simply removed all the unnecessary lines and text until I had just footprint left over. I then set my grid to 1.27 (5 mil) and Snap Mode = Fortieth Grid. I selected the image and moved it under maximum zoom until my pins lined up perfectly to the grid.

clip_image014

All that’s left to do is to place the pads and it’s done!

clip_image016

I am very chuffed with my handywork. What could otherwise have taken hours can now be done in minutes if you have a suitable PDF or DXF datasheet and a little bit of skill and patience.

In case you were wondering how I got the Fez Panda board imported… Well that was even easier. The folks at GHI Electronics supplies the board layout and schematics in Eagle format. I simply followed the instructions in the DesignSpark help file for importing Eagle files and voila!

How cool is this? A 3D view of my board – and I didn’t have to do a thing extra to get it! Next I’ll figure out how to make the LCD look real.

clip_image018

There are some catches though (aren’t there always?)…

First, all the colours are different between Eagle and DesignSpark and it is hard to visually compare the two boards to check if they are exactly the same. I’m going to have to trust it.

Next, after importing the schematic and forward-annotating the imported PCB, it picked up a bunch of differences between the parts on the schematic and those on the board. If I allow it to update the board with the latest parts, it messes up some of the footprints and tracks – so not a good idea. I don’t know how to resolve that yet and I don’t intend to re-map every component thank you very much. I’m just going to have to use the board as a PCB part and ignore the schematic for now.

So there you have it. From PDF to component – from Eagle to DesignSpark. I think I have a new friend.

Categories: DesignSpark

Mini Lathe

If there’s one piece of equipment I always miss when I build projects is a lathe. If only I could make that coupler to connect the stepper to the lead screw… Well now I can!

I really spoiled myself and got a mini lathe. It’s a tiny desktop model, but just fine for the work that I need to do.

Here it is…

IMG_1403

Living at the coast has some challenges with rust, so I decided to make a box to protect the lathe – with a little extra feature you’ll see in a moment.

The idea is to use a large plastic container upside down as a lid.

First I put a wooden base in the lid with the drip tray attached.

IMG_1405 IMG_1406

The special feature… A light mounted on the inside of the container.

IMG_1408

The new lid very conveniently lifts up and the light shines nicely over the work area.

IMG_1409

You can see the toolbox behind the lathe – and by sheer luck fits perfectly between the lathe and the lid when closed.

IMG_1410

And here’s the final product… safe and sound (and dry!)…

IMG_1412

So what to build?

I already measured the positions for the stepper motors to turn this baby into a CNC machine. Along with the traditional x and z movement, I plan to get a y movement by turning the chuck with a stepper motor or the drive motor itself. By using a bit of nasty polar coordinates, there’s no reason the “table” can’t be sitting on the chuck and I should be able to do some horizontal milling or drill PC boards. More on that later.

Categories: Uncategorized

From Creativity to Innovation

I just got back from the second course on creativity and innovation by Kobus Neetling. I really found this so inspiring and I must thank my employer for giving me this opportunity to attend what would otherwise not fall into the category of “IT training”. It was an energising and fulfilling experience to say the least.

In the first course we learnt that people have brain profiles that control their preferences and behaviour. After being enlightened about this fact, I can all of a sudden look at people with new eyes and new respect for our differences and strenghts as human beings. Having a “whole brain approach” to tackling problems, creating teams and even developing presentations and speeches is fascinating – and so elegantly simple once you understand the mechanics of the brain.

I obviously can’t repeat the contents of the entire course here, but the things below struck me as something more people should know about.

For starters, Neetling classifies the brain into 4 quadrants and 8 dimensions. The quadrants are:

  • L1 (Left Brain): Realist/Analyst
  • L2: (Left Brain): Preserver/Organiser
  • R1: (Right Brain): Strategist/Imagineer
  • R2: (Right Brain): Socialiser/Empathiser

People’s preferences and to an extent, behaviour, are controlled by the dominance of one or more of the dimensions.

It explains a lot about myself – and those I tend to work well with or butt heads with.

Realists prefer:

  • Clarity of thinking
  • Concrete information
  • To focus on a specific goal or outcome
  • No distractions
  • Achievable and clearly defined goals
  • No clutter and confusion
  • To weigh pros and cons
    To understand all possible consequences

Analysts prefer:

  • To get to the essence of things
  • To dig deeper to achieve results
  • To be involved in financial and investment matters
    To connect a figure or measurement to things
  • Certainty
  • To calculate, probe, research and examine conditions

 Stalwarts prefer:

  • Circumstances where traditions and well-proven methods and practices are respected
    Rules and regulations to be in place
  • To work in a mehodical and cautious manner
  • A neat and secure environment
    To work with skilled, loyal and trustworthy people

Organisers prefer:

  • To plan, organise and arrange things
  • To follow an orderly, detailed and systematic approach
  • To implement and to put things into action
  • To work according to a schedule or “to do” list
  • To administer and oversee tasks
  • Priorities such perserverence, effectiveness and usefulness

Socialisers prefer:

  • To work with others in groups
  • To spend free time in the company of others
  • To be surrounded by people and be part of gatherings and crowds
  • To bring people together at work or when entertaining
  • To share information
  • To reach consensus and to encourage others to participate in sharing ideas
  • To be outgoing and energetic when in the company of others

Empathisers prefer:

  • To assist, help and reach out to others
    To serve others through caring, sensitive and unobtrusive attitude
  • To depend on your intuition
  • To encourage others to achieve
  • A positive, hopeful and caring environment
    The companionship of a few friends and family (or animals!)
  • To place the needs of others above your own

Strategists prefer:

  • To connect past and future
  • To see the vision, even to make forecasts and predictions
  • To challenge existing approaches and to ask the Why? questions
  • A challenge and therefore unfamiliar territory and new experiences
    To consider a variety of possibilities
  • Not to shy away from risk when you are involved in designing future trends

Fantasisers prefer:

  • To think in pictures
  • To use metaphors and images to describe experiences
    To draw pictures, doodle and scribble when listening to someone
  • To play around with impossible ideas
  • To daydream, fantasise and think beyond the ordinary
  • To be reflective and meditative
  • To often be unsympathetic and to do things “your way”
  • To nurture “strange” ideas

Do you recognise yourself in any of these? Sure you do… You’re just as much the product of a chemical bonding process between neurons as I am! Most people have a combination of all of these, but there are dominant dimensions. Very interesting science behind it!

My dominant profile is Strategist + Analyst, which just happens to be a neat fit to by career! I am also a bit more conservative and organised than the usual Right brain-dominant person, so I can get a job done – but beware… not in a strictly procedural manner as an L2 (highly methodical) person would like it. I drive project managers nuts with my “scatter brain” and I often have to convince people to trust that there’s method in my madness. At least now I can explain it somewhat.

The rest of the course focused on techniques for using all the dimensions of the brain to first be creative (come up with new ideas) and turn that into innovation (putting great ideas into action). The energy in the room was amazing and the ideas that came out of our group suprised even ourselves. I really hope you’ll see some of these revolutionary new concepts in the financial services industry in the year to come if we can turn these ideas into products or services.

Some quotes I found inspiring…

“Creativity is thinking up new things. Innovation is doing new things.” Theodore Levitt

“Creativity is taking creative ideas and bringing them into the world so that they change lives, and so they also change the organisations that bring them into the world.” Arnold Wasserman

and the one I’m sure to torment my colleagues at work with starting immediately…

“The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man.”

I’ll be unreasonable from now on thank you. Gotta love it.

Hello World (I’m back!)

May 5, 2011 1 comment

Well, here we go! After snoozing and losing my old blog in the Live Spaces shut-down, I will be building from scratch here.

Just finished another entry for Embedded Spark 2011 challenge. This time it was really hard because there were so many pressures at work and disruptions at home. I’m pretty chuffed with seeing the concept of a wearable computer, controlled by gestures come to life. I just wish I could have added some more of the advanced features like video streaming and augmented reality. It’s going to be hard to match the success of RoomBox, but you have to be in it to win it – so we’ll see what happens.

Categories: Uncategorized Tags: