3d Printing update: Sprinter versus Marlin firmware & more

The Marlin vs Sprinter Debate isn’t much of a debate. The instant I upgraded from Sprinter to Marlin for our Reprap Prusa the difference was worth it. Marlin is Sprinter with the ability to look ahead for smoother acceleration and g2 & g3 gcode support. Here is the same object printed twice, once with Marlin (left) and once with Sprinter (right).

These were printed with the same gcode

I sliced them with the newest version of Slic3r .7 and printed them with Pronterface. Next I tried turning on arc support in slic3r but found that some prints printed beautifully while others spiraled inwards. Here is a z-axis stablizer that show both some nice arcs and some than crawled there way inward. The piece on the right is with Marlin with arc support turned on and the piece on the left was with Sprinter. If you look closely at the bottom of the piece on the right you will see the issue. Since are pieces aren’t too curvy I think I’ll be keeping arc support turned off until it is out of the experimental phase.

The last update for the printer is we will be switching from our pla bearings to some nice lm8uu ones. Here are the x-ends. Hopefully they give us even cleaner prints.

Figuring out why Node.js modules won’t load

We ran into some problems getting our modules to load, so wanted to share a couple of things that may help other C++ module developers on node.js.

Won’t load
If your module won’t load, you are going to get a very terse message, which simply says this or similar:

ed@computer$ ../node-0.7.0/node ./test.js
node.js:218
throw e; // process.nextTick error, or 'error' event on first tick
^
Error: Unable to load shared library /somedir/yourmodule.node

Now, the failure to load could be:

  • The module is not in the right directory (see below)
  • Your module has unresolved symbols
  • You are using libraries, probably custom ones, dlopen() can’t find.

This is just like any other shared library. Unfortunately, node.js won’t tell you much on that. This is because another function, dlerror() has most of the information you need, is not called by DLOpen() – node’s native code for shared object loading on Linux.

Quick patch
If you need more information, you can patch node.cc in the src/ at the top of the tree. Locate the node.cc, and make this change in DLOpen()  (approx line 1645-ish). This code would really belong in deps/uv/src/unix/dl.c – but this is not a permanent fix, and I didn’t want to modify it, and then also have to modify this function. Bottom line – this is for Linux people only.

if (err.code != UV_OK) {
Local<Value> exception = Exception::Error(String::Concat(String::New("Unable to load shared library: "), String::New(dlerror())));
// WigWag: Not having output of the actual dlerror makes developing native modules harder. Need this.
// Local<Value> exception = Exception::Error(
// String::Concat(String::New("Unable to load shared library "),
// args[0]->ToString()));
return ThrowException(exception);
}

Also add this at the top of node.cc, and then rebuild node.

#include <dlfcn.h>

Now you will get something a bit more meaningful, such as an undefined symbol:

Error: Unable to load shared library: /home/ed/workspace/node/lib/mymodule.node: undefined symbol: _somenamemangledsymbol_nameE

Discovering the search path
Node will look in a bunch of places for your native module. It’s search area is a combination of its own path resolution and the behavior of the system call dlopen() called in the libuv. If it can’t find it, you’ll see something like this:

node.js:218
        throw e; // process.nextTick error, or 'error' event on first tick
              ^
Error: Cannot find module 'yourmodulename'

So if your library won’t load, and your sure you put it in the right place, use strace. Strace hooks the system calls a program makes, and prints their output. Luckily, almost any file reading is done with a system call.
Example:
Here is a test program trying to do a require mymodule=(‘mymodule’);

ed@computer$ strace ../node-0.7.0/node ./zdb.js 2>&1 | grep ENOENT
...
stat("/home/ed/.node_modules/mymodule", 0x7fff6aa7b488) = -1 ENOENT (No such file or directory)
stat("/home/ed/.node_modules/mymodule.js", 0x7fff6aa7b418) = -1 ENOENT (No such file or directory)
stat("/home/ed/.node_modules/mymodule.json", 0x7fff6aa7b418) = -1 ENOENT (No such file or directory)
stat("/home/ed/.node_modules/mymodule.node", 0x7fff6aa7b418) = -1 ENOENT (No such file or directory)
open("/home/ed/.node_modules/mymodule/package.json", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/home/ed/.node_modules/mymodule/index.js", 0x7fff6aa7b418) = -1 ENOENT (No such file or directory)
stat("/home/ed/.node_modules/mymodule/index.json", 0x7fff6aa7b418) = -1 ENOENT (No such file or directory)
stat("/home/ed/.node_modules/mymodule/index.node", 0x7fff6aa7b418) = -1 ENOENT (No such file or directory)
stat("/home/ed/.node_libraries/mymodule", 0x7fff6aa7b488) = -1 ENOENT (No such file or directory)
stat("/home/ed/.node_libraries/mymodule.js", 0x7fff6aa7b418) = -1 ENOENT (No such file or directory)
stat("/home/ed/.node_libraries/mymodule.json", 0x7fff6aa7b418) = -1 ENOENT (No such file or directory)
stat("/home/ed/.node_libraries/mymodule.node", 0x7fff6aa7b418) = -1 ENOENT (No such file or directory)
open("/home/ed/.node_libraries/mymodule/package.json", O_RDONLY) = -1 ENOENT (No such file or directory)
...

That funky stuff at the end of the command there, redirects stderr to stdout (strace outputs to stderr), and then we grep on ENOENT, because that is the error for a missing file. To get a better idea, drop the redirection and grep, and just run the command. You will see every system call node makes. Now you know exactly where node looked for your native module. This same technique could apply to a bunch of other debugging issues.

If you are still having issues with node finding a custom shared library you wrote, go investigate LD_LIBRARY_PATH and -Wl,-rpath switch in gcc. Or see this HOWTO.

 

Building C++ modules on node.js 7.0

I did a bunch of Googling this morning looking for how to build a C++ node.js module, using the new gyp build system. There a plenty of blog posts on doing native module builds via node-waf but not with the gyp_addon python script.

I took the liberty of basing this example off Teemu Ikonen’s blog entry on threaded extensions.

Here is the C++ node.js module source. This is an extension which creates a thread, and then makes a callback every 5 seconds. It’s a good example because it also uses some of node’s internal libraries (namely libev):

GYP build system: gyp_addon
Go ahead and look at the original blog entry if you want to build with node-waf. Going forward, however, with new versions of node you will need to use the gyp build system. Luckily, there is a great project on github which will make this easier depending on your circumstances. This is TooTallNate’s node-gyp script, which itself uses node to make cross-platform building of your extension easier. I highly suggest going this route if

  • You are developing a standard extension for the node.js system
  • You are not doing cross compiling
  • and you want cross platform support (who wouldn’t)

On the other hand, if you are working off the tip and/or are doing cross-compiling, or some other case where you need a custom node.js source tree itself, then you may want to go this route… First you need a gyp build file. Here is one for the texample.cpp extension. More description follows.

Now, we are doing a few extra things here:

  • We are specifying a build for x86_64 arch. See target_arch line in variables. (For some reason node pays no attention to config.gypi in the root of the source folder. Not sure why… post something if you know.)
  • And to also support x86_64 we have a line ['cflags'] line under Linux section specifying -fPIC. You need this option when building shared libraries on x86_64 for node.
  • Notice node_root, a variable defined at the top. This is where the node.js source is in comparison to your source files
  • We also include some non-standard include directories, specifically stuff used inside node. deps/uv/include/uv-private is where the libev headers are. And this example uses those.

Building
For gyp_addon to work, you will need to run it out of the tools directory, unless you have node more permanently installed. Since texample.gyp references everything relative to where it (texample.gyp) is, then this will be fine. Just point gyp_addon to it:

ed@computer:~/node-0.7.0/tools$ ./gyp_addon ../../src/texample.gyp
ed@computer:~/node-0.7.0/tools$ make
  ACTION Regenerating Makefile
  CXX(target) out/Debug/obj.target/texample/../../src/texample.o
  SOLINK_MODULE(target) out/Debug/obj.target/../../src/texample.node
  SOLINK_MODULE(target) out/Debug/obj.target/../../src/texample.node: Finished
  COPY out/Debug/texample.node
ed@computer:~/node-0.7.0/tools$ ls out/Debug/
linker.lock  obj.target  src  texample.node

and your output should be similar.

At some later point we will show an example of cross-compiling using this method.

Quick test:

ed@computer:~/node-0.7.0$ ./node
> texample = require('./texample');
{ start: [Function] }
> texample.callback = function(i) {
... console.log("bang: " + i );
... }
[Function]
> texample.start();
true
> bang: 0
bang: 1
bang: 2
bang: 3

“Unable to open shared library”
If you get such an error, and you know for certain that node.js really is finding your library file (you could verify with strace), its likely that your object format’s are not compatible. This means you built the node.js for x86_64 and your object file is ia32. You can verify this by doing something similar to:

ed@computer:~/node-0.7.0/tools$ objdump -g out/Debug/texample.node
out/Debug/texample.node:     file format elf32-i386

This command also run on the node executable needs to output the same format, here elf32-i386. If you’re on 64-bit linux, it would be elf64-x86-64. Putting this here because I suspect folks running node on x64 will run into this issue.

Arduino Project (Bell Enclosure Part 1)

So I’ve been printing away many different projects and fine tuning my Reprap 3d Printer when my wife asks me when I am going to get her bell working again. A while back I made a bell that rings whenever someone visits her blog. We reorganized our home office and the ardruino, servo and bell that were literally screwed into the desk and wall had to get taken down.

Well this time I am going to make the whole set up a lot more portable and enclosed with the use of my 3d printer. In about 2 hours I designed and printed this enclosure:

Arduino Enclosure

Arduino Enclosure 2

The design is a little snug and I plan on modifying it and putting it on thingyverse.com in the near future. You may notice there isn’t a top yet. That is part of the plan. I now an going to design and print a top that will allow me to mount the bell and servo and attach it to my wall with only one screw. I figure this base can be used in other arduino projects that follow.

Merging source code with Meld

Merging source code from one repository tree to another can be a daunting task.  At WigWag we utilize the Contiki OS, “The Operating System for the Internet of Things,”  for all of our microcontroller devices.  Contiki is under constant development with daily changes and therefore every once in a while we like to merge in the latest work from the Contiki team, with our source repository.  With Meld, this merging effort is very simple.

Installation
If you are on Ubuntu, installing Meld is simple.

$ sudo apt-get install meld

A great list of similar tools for your OS can be found on this Stackoverflow article.

Setting up Meld for the merge

Meld source selection and target

  • Open Meld and chose File New
  • Select the “Directory Comparison” in the center tab
  • Point your source to “Mine” and the other source to “Original”… Press OK

Understanding the layout

meld window mine and target source

  • On the right is our source tree “Mine”
  • On the left is the new version of “contiki” just freshly checked out using GIT
  • All the files are displayed in a tree.   Colors represent the differences.
  • Green is a new file
  • Grey crossed out is a non-existing file
  • Red represents a file with a delta

Understanding the toolbar

meld toolbar

  • Up and down arrows “go to the next difference”
  • The left arrow pushes changes form “mine” to “contiki” –an operation we don’t need at this time
  • The right arrow pushes changes from “contiki” to “mine”
  • delete will delete a file
  • Hide will hide a file
  • Case will turn on and off case sensitivity

The Merging operation

  • Simply start at the top of the tree, and scroll down through the new files, missing files and deltas
  • Chose which files to merge based on your merging preferences.  (See our merging rules below)

Merging Deltas

Meld file delta red highlight

  • When encountering a delta you will see a red file on both sides
  • Double click the delta
The differences show up highlighted on the left and right
  • The comparison window will show you the differences between the left and right
  • You can push each difference into the other side by clicking the arrows
  • If you make a mistake, UNDO works just fine…. by the way this is a full editor too.  So you can also just edit the code in place
copy in text from on side to the other hold control
  • by holding down the control key, you can “copy text” from one side to the other, below or above the other source.
  • to actually complete the push using linux, you have to press ctrl-shift click

Our rules for merging

While we are still in a very early development stage, we follow a fairly loose merge policy.  clearly once we start shipping product, our merge rules will become much more stringent.

  • New Contiki file –> Push into Mine
  • Changed Contiki file, no WigWag edits in Mine –> Push into Mine
  •  changed Contiki file, with WigWag edits in Mine –> Push the delta lines that don’t effect our code, where it does effect our code, push in the contiki changes and comment them out

Quick demo video

  • to see the whole thing in effect, watch this short video

Using ICU4C Regex test program

If you need Unicode support, your goto library for the real deal is ICU. ICU also has very solid regex support. And while ICU has plenty of documentation, examples on the net sometimes fall short. The following is a simple command line program which will do both regex replacement, matching and capture groups. It also serves as a decent example of how to use the library. ICU is the basis for unicode and regex support in Android and iOS as well. So this little test program will help you there as well.

ICU likes to do everything in UTex, which I think is always UTF-16. So, to do UTF-8 or ASCII you need to conversion. The program converts the UTF-8 on the command line, runs it through the regex and returns.

Usage

Since you are running this on the command line, you need to escape ‘\’ some chars in bash, mainly backslashes and exclamation marks. It will printout the regex it actually used.

Source on gist. (Easier to copy)

$ ./regextest "Hello(.*)" "Hello Bob"
pattern:     -->[Hello(.*)]<--
test on:     -->[Hello Bob]<--
MATCH: string matches regex
Capture group 0: -->[Hello Bob]<--
Capture group 1: -->[ Bob]<--

$ ./regextest "(ftp|http|https):\\/\\/(\\w+:{0,1}\\w*@)?(\\S+)(:[0-9]+)?(\\/|\\/([\\w#\!:.?+=&%@\!\\-\\/]))?" http://www.google.com

pattern:     -->[(ftp|http|https):\/\/(\w+:{0,1}\w*@)?(\S+)(:[0-9]+)?(\/|\/([\w#\!:.?+=&%@\!\-\/]))?]<--
test on:     -->[http://www.google.com]<--
MATCH: string matches regex
Capture group 0: -->[http://www.google.com]<--
Capture group 1: -->[http]<--
Capture group 2: -->[]<--
Capture group 3: -->[www.google.com]<--
Capture group 4: -->[]<--
Capture group 5: -->[]<--
Capture group 6: -->[]<--

Source

// The following code is freeware:
// regextest.c
// Author: ed@wigwag.com
//
// Simple test program for regex using ICU's regex matching.

#include <stdio.h>
#include <getopt.h>
#include <string.h>
#ifdef __cplusplus
#include <cstdlib>
#endif
#include <unicode/regex.h>
#include <unicode/utext.h>
#include <unicode/errorcode.h>
#include <unicode/ucnv.h>
#include <unicode/utypes.h>
#include <unicode/uchar.h>

static const char OptionsInfo[] = "Usage: regextest [-r] \"PATTERN\" \"TEST STRING\"\n"
								  "-r : replace mode -> \"PATTERN\" \"TEST STRING\" \"replacement\"\n"
								  "You need to escape these items like this: \\\" and \\$ b/c of bash.\n"
								  "\n";

/* ICU library libs/programs...
 * http://www.linuxfromscratch.org/blfs/view/svn/general/icu.html
 * http://icu-project.org/download/4.4.html#ICU4C
 *
 */

int main(int argc, char *argv[]) {
	int exitcode = 1;
	bool replace_mode=false;
    int c;
    int digit_optind = 0;

    while (1)
      {
        c = getopt (argc, argv, "r");
        if (c == -1)
        	break;

        switch (c)
          {
          case 'r':
        	  replace_mode = true;
        	  break;
          }
      }

	if(argc-optind < 2) {
		printf("%s",OptionsInfo);
		exit(1);
	}
	if(replace_mode && argc-optind < 3) {
		printf("%s",OptionsInfo);
		exit(1);
	}

	printf("pattern:     -->[%s]<--\n", argv[optind]);
	printf("test on:     -->[%s]<--\n", argv[optind+1]);

	UErrorCode        status    = U_ZERO_ERROR;
	UText *regex1 = NULL;
	UText *matchthis = NULL;

	//	static const char *regex_validate_string = "(?:cp\\:([0-9]+)\\:){0,1}\"(.*)\"";

	regex1 = utext_openUTF8(regex1, argv[optind], -1, &status);
	//regex1 = utext_openUTF8(regex1, regex_validate_string, -1, &status);
	matchthis = utext_openUTF8(matchthis, argv[optind+1], -1, &status);

	RegexMatcher *matcher = new RegexMatcher(regex1, 0, status);
	if (U_FAILURE(status)) {
		// Handle any syntax errors in the regular expression here
		printf("Syntax error in regex?\n");
		icu::ErrorCode ec;
		ec.set(status);
		printf("Error was: %s\n",ec.errorName());
		utext_close(regex1);
		utext_close(matchthis);
		delete matcher;
		exit(exitcode);
	}

	if(!replace_mode) { // MATCH TEST

		//	UnicodeString    stringToTest = "Find the abc in this string";
		matcher->reset(matchthis);

		if (matcher->matches(status)) {
			// We found a match.
			printf("MATCH: string matches regex\n");
			//	   int startOfMatch = matcher->start(status);   // string index of start of match.

			if(matcher->groupCount() > 0) {
				UConverter *conv = ucnv_open("US-ASCII", &status);
				for(int x=0;x<=matcher->groupCount();x++) {
					//				UText *grp = NULL;
					UnicodeString US = matcher->group(x, status);
					if (U_FAILURE(status)) {
						icu::ErrorCode ec;
						ec.set(status);
						printf("Error was: %s\n",ec.errorName());
					} else {
						UChar *out = (UChar *) malloc(1000);
						char *outcs = (char *) malloc(1000);
						US.extract(out,1000,status);
						ucnv_fromUChars(conv,outcs,1000,out,u_strlen(out),&status);
						printf("Capture group %d: -->[%s]<--\n", x, outcs); // works as long as UTF8
						//					printf("Capture group %d: %s\n", x, grp); // works as long as UTF8
						if(out) free(out);
						if(outcs) free(outcs);
					}
				}
			}

			exitcode = 0;
		} else {
			printf("FAIL: no match\n");
			exitcode= 2;
		}

	} else { // REPLACER TEST
		UText *replacedtxt = NULL;
		UText *replacement = NULL;
		replacement = utext_openUTF8(replacement, argv[optind+2], -1, &status);
		matcher->reset(matchthis);
		printf("replacement: -->[%s]<--\n", argv[optind+2]);

		replacedtxt = matcher->replaceAll(replacement,replacedtxt,status);
		if (U_FAILURE(status)) {
			// Handle any syntax errors in the regular expression here
			printf("Syntax error in regex?\n");
			icu::ErrorCode ec;
			ec.set(status);
			printf("Error was: %s\n",ec.errorName());
			utext_close(regex1);
			utext_close(matchthis);
			utext_close(replacement);
			delete matcher;
			exit(exitcode);
		}

		if(replacedtxt) {
			// Replacement did something...
			printf("REPLACE: string replaced...\n");
			//	   int startOfMatch = matcher->start(status);   // string index of start of match.
			UChar buf[500];
			printf("new length: %ld\n", utext_nativeLength(replacedtxt));
			utext_extract(replacedtxt,0,500,buf,500,&status);

			if (U_FAILURE(status)) {
				// Handle any syntax errors in the regular expression here
				printf("Syntax error in extraction:\n");
				icu::ErrorCode ec;
				ec.set(status);
				printf("Error was: %s\n",ec.errorName());
				utext_close(regex1);
				utext_close(matchthis);
				utext_close(replacedtxt);
				utext_close(replacement);
				delete matcher;
				exit(exitcode);
			}

			// Yawn... and convert to ascii.  Can we make it any more complicated??
			UConverter *conv = ucnv_open("US-ASCII", &status);
			char *outcs = (char *) malloc(1000);
			ucnv_fromUChars(conv,outcs,1000,buf,u_strlen(buf),&status);

			// Also, use function u_strFromUTF8 / u_strToUTF8

			if (U_FAILURE(status)) {
				// Handle any syntax errors in the regular expression here
				printf("Syntax error in extraction:\n");
				icu::ErrorCode ec;
				ec.set(status);
				printf("Error was: %s\n",ec.errorName());
				utext_close(regex1);
				utext_close(matchthis);
				utext_close(replacedtxt);
				utext_close(replacement);
				free(outcs);
				delete matcher;
				exit(exitcode);
			}
			ucnv_close(conv);

			printf("Replaced result: -->[%s]<--\n", outcs); // works as long as UTF8
//			printf("Capture group %d: %s\n", x, grp); // works as long as UTF8
			if(outcs) free(outcs);
	utext_close(replacedtxt);
		} else {
			printf("Result: No replacement.\n");
		}

		utext_close(replacement);

	}
	utext_close(regex1);
	utext_close(matchthis);
	delete matcher;
	exit(exitcode);
}

Build

$ gcc regextest.cpp -licui18n -licuuc -licudata -o regextest

Techshop and 3d Printing

So after months of horrible prints.  I decided to join the bay area reprap google group.  The group meets at Techshop in Menlo Park and Noisebridge in San Francisco.  (More to come on Techshop on a later post).  The group is basically a group of guys who meet up weekly to work on their 3d printers and actually print some stuff.  After two meetings I have went from this:

To this:

(Don’t mind the crooked part at the top, that was a human error, not a printer error)

I don’t want to get too cocky but one of the members told me it was “F’n unbelievable” that I was able to get a working machine without any support.  While others definitely made me feel like a total nOOb.  My main issue was my extruder.  I ordered my nozzle from Mixshop and thought it was fine.  One of two things were wrong with it, it was either the shape of the tip (slightly concave at the end) or the hole just stripped out, cause my extrusion to be 1.5 mm.  It was supposed to be .5 mm.

A new heater block and nozzle borrowed from a new friend and I was in business.  While a few upgrades are to come, like a brutstruder from Makergear for my coldend, I am confident that I will be able to be print excellent prototypes in the months to come.

Here is a video of my printer in action, enjoy:

 

CES (2): Liquipel and ZigBee cameras

Here a couple of more interesting things from CES: Liquipel, the hyped ‘nano’ coating for smartphones and potentially other electronic devices, as well as an 802.15.4 camera.

Liquipel

I ran across Liquipel on the second day of the show. Not incredibly well known at the time, they had a small booth at the show, and below are a couple of videos. Since then they have gone viral, and actually sold out of product for a while. Liquipel puts a very thin (microscopic?) coating on a device which prevents it from shorting out when dropped in water. It does work – I played with it myself at the show. You cannot tell the device is coated. However, a couple of questions remain:

  • Will it work well if dropped in dirty/heavy water (non purified water, with either particles or heavier metals in it)? Some speculation on the web says it will not.
  • And how well will it really hold up over time? Liquipel claims it will outlast the life of your device.

If anyone has experience with this, please comment below.

And here is Liquipel on a piece of tissue paper, notice how it essentially causes the water to bead up as if the paper was now a very polar surface…

802.15.4 QVGA camera

We hadn’t seen one of these yet, but did know a couple were out there. This is a camera running a QVGA (320×240) resolution picture at about 22fps. 802.15.4 has a theoretical max of about 250kbps (that’s bits per second). Real world performance is more like 120-180kpbs. Not exactly impressive. Some hardcore details can be hard here in a technical research paper from Intel. So this makes this little item all the more impressive. We assume this is a MPEG-4 stream, which could easily compress a QVGA stream down to under 200kbps. But the fact that it operates smoothly, and does so in the horrible RF space at CES is impressive as well.

You can see it did drop a number of frames during that high motion. That’s probably the compression catching up.

Cross-compile node.js for ARM

Note: the information was current as of node 0.7.0 top of tree, from circa Jan 21, 2012. So modify as necessary.

There are three major steps/challenges in getting node.js to compile on ARM

  • We need to get V8 to compile on your target arch. Look at our previous article.
  • Need to get all the node.js other dependencies to use the right toolchain.
  • Make sure we tell node’s dependencies where the libs and includes are for your cross-environment.

We are going to assume that you have successfully got V8 compiling for your hardware. If not see Cross-compile V8 for ARM.

Understanding the node.js build process.

As of 0.6.x, node.js uses gyp (Generate Your Projects) from the Chromium project. GYP can generate Visual Studio, XCode projects, and on Linux Makefiles. For node, .gyp files located in the root and /deps dirs of the source tree direct generation of makefiles.

node is configured using a python script, configure, in the root of the source. Building node for execution on the build host is very easy (all examples are for Linux)

$ ./configure
{ 'target_defaults': { 'cflags': [],
                       'defines': [],
                       'include_dirs': [],
                       'libraries': ['-lz']},
  'variables': { 'host_arch': 'ia32',
                 'node_install_npm': 'true',
                 'node_install_waf': 'true',
                 'node_prefix': '',
                 'node_shared_cares': 'false',
                 'node_shared_v8': 'false',
                 'node_use_dtrace': 'false',
                 'node_use_isolates': 'true',
                 'node_use_openssl': 'true',
                 'node_use_system_openssl': 'false',
                 'target_arch': 'ia32',
                 'v8_use_snapshot': 'true'}}
creating  ./config.gypi
creating  ./config.mk
$ make

configure has an option for cross compile, –dest-cpu=ARCH where it says arm, ia32 and x64 are valid architectures. However, you may be asking… great, but how does it know where my cross compile toolchain is? Well, it doesn’t. This is not a cross-compile option, just an architecture option.

Full cross compile support does not really seem to be baked into node.js yet. In order to cross-compile we are going to override some build vars, run the configure script, and then open a shell where we can build and do any other make targets we want. Below is a script which sets up a cross compile, using a specified toolchain, on Linux. However, your mileage may vary. Modify for your own purposes and try the script out. If you get some issues, then read on.

setup-cross.sh

#!/bin/sh -e

# some vars to specifcify our toolchain. Don't need to export
CROSS_BASE=/opt/freescale/usr/local/gcc-4.4.4-glibc-2.11.1-multilib-1.0/arm-fsl-linux-gnueabi
CROSS_INCLUDE=${CROSS_BASE}/arm-fsl-linux-gnueabi/multi-libs/usr/include/
CROSS_CC=${CROSS_BASE}/bin/arm-fsl-linux-gnueabi-gcc
PREFIX_BIN=$CROSS_BASE/bin/arm-fsl-linux-gnueabi

export CSTOOLS=$CROSS_BASE/bin

# cross library directory, should include stdc++:
export CSTOOLS_LIB=/opt/ltib/rootfs/lib
export CSTOOLS_USR_LIB=/opt/ltib/rootfs/usr/lib

# libc & system headers:
export CSTOOLS_INC=${CROSS_BASE}/arm-fsl-linux-gnueabi/multi-libs/usr/include/

export TARGET_ARCH="-march=armv5te"
#ARM9? try these: export TARGET_TUNE="-mtune=cortex-a8 -mfpu=neon
#       -mfloat-abi=softfp -mthumb-interwork -mno-thumb" # optional
export TARGET_TUNE="-mtune=arm926ej-s -mfloat-abi=soft -mno-thumb-interwork"

export TOOL_PREFIX=$PREFIX_BIN
export CCFLAGS="-march=armv5te -mtune=arm926ej-s -mno-thumb-interwork -lstdc++"
export ARM_TARGET_LIB=$CTOOLS_LIB
export CPP="${PREFIX_BIN}-gcc -E"
export STRIP="${PREFIX_BIN}-strip"
export OBJCOPY="${PREFIX_BIN}-objcopy"
export AR="${PREFIX_BIN}-ar"
export F77="${PREFIX_BIN}-g77 ${TARGET_ARCH} ${TARGET_TUNE}"
unset LIBC
export RANLIB="${PREFIX_BIN}-ranlib"
export LD="${PREFIX_BIN}-ld"
export LDFLAGS="-L${CSTOOLS_USR_LIB} -L${CSTOOLS_LIB} -Wl,-rpath-link,${CSTOOLS_LIB} \
  -Wl,-O1 -Wl,--hash-style=gnu"
export MAKE="make"
export CXXFLAGS="-isystem${CSTOOLS_INC} -fexpensive-optimizations \
   -frename-registers \
   -fomit-frame-pointer -O2 -ggdb3 -fpermissive -fvisibility-inlines-hidden"
export LANG="en_US.UTF-8"
export HOME="/home/ed"
export CCLD="${PREFIX_BIN}-gcc ${TARGET_ARCH} ${TARGET_TUNE}"
export PATH="${CSTOOLS}/bin:/opt/code-sourcery/arm-2009q1/bin/:\
${HOME}/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games"
export CFLAGS="-isystem${CSTOOLS_INC} -fexpensive-optimizations \
     -frename-registers \
     -fomit-frame-pointer -O2 -ggdb3"
export OBJDUMP="arm-none-linux-gnueabi-objdump"
export CPPFLAGS="-isystem${CSTOOLS_INC}"
export CC="${PREFIX_BIN}-gcc ${TARGET_ARCH} ${TARGET_TUNE}"
export TERM="screen"
export SHELL="/bin/bash"
export CXX="${PREFIX_BIN}-g++ ${TARGET_ARCH} ${TARGET_TUNE}"
export NM="${PREFIX_BIN}-nm"
export AS="${PREFIX_BIN}-as"

# Configure node.js for cross-compile

# Try this first: ./configure --without-snapshot --dest-cpu=arm --gdb
./configure --dest-cpu=arm --gdb --shared-v8

bash --norc

For this script to work, I ended up needing to make some minor changes to node 0.7.0. For one, my build target is not an ARM 7. Two, my cross libraries are in a couple of locations. Also, I wanted to build with snapshot, and the cross-compile in my testing failed unless –without-snapshot was used on configure. Below are the tweaks:

Again, bear in mind this was top of tree, node 0.7.0 in Jan 2012, so you should check your source.

Correct floating point options, –march, etc.

In our other article on V8 we discussed how to choose the right options for your ARM arch toolchain. node’s configure script doesn’t allow you to set these, but our setup-cross.sh will set the options for building most of the support binaries. But the V8 configure will need more information… basically by default, node will try to build V8 for an ARM-7 architecture (with NEON and VFP3). How do we change this? In deps/v8/tools/gyp/v8.gyp you will notice a snippet:

# http://code.google.com/p/v8/issues/detail?id=914
     'conditions': [
        ['armv7==1', {
         # The ARM Architecture Manual mandates VFPv3 if NEON is
         # available.
        }],
       ],
...
      },{ # else: armv7!=1
         'variables': {
            'mksnapshot_flags': [
            '--noenable_armv7',
            '--noenable_vfp3',
 ],

So we need armv7 to != 1. Since I could not find an configure option to fix this, I just fixed it with a small mod in the configure script.

def configure_node(o):
  # TODO add gdb
  o['variables']['node_use_isolates'] = b(not options.without_isolates)
...
  o['variables']['armv7'] = "0"  # add this if you are not >= armv7

Cross compile with snapshot

Take a glance at deps/v8/tools/gyp/v8.gyp at about line 90.

  {
          'target_name': 'v8_snapshot',
          'type': '<(library)',
          'conditions': [
            ['want_separate_host_toolset==1', {
              'toolsets': ['host', 'target'],
              'dependencies': ['mksnapshot#host', 'js2c#host'],
            }, {
              'toolsets': ['target'],
              'dependencies': ['mksnapshot', 'js2c'],
            }],
            ['component=="shared_library"', {
              'conditions': [
                ['OS=="win"', {
                  'defines': [
                    'BUILDING_V8_SHARED',
                  ],
                  'direct_dependent_settings': {
                    'defines': [
                      'USING_V8_SHARED',
                    ],
                  },

If we want snapshot support, we need that mksnapshot#host. Othwerwise your cross compile will eventually fail, because it will not be able to run mksnapshot on the host (as it was compiled for the target). So we need want_separate_host_toolset to == 1.

We can fix this, again, by just adding a few lines into the configure script. Find the configure_node(o) function in configure, and add these lines:

def configure_node(o):
  # TODO add gdb
  o['variables']['node_use_isolates'] = b(not options.without_isolates)
...
 if options.dest_cpu:                                     # add these two lines
    o['variables']['want_separate_host_toolset'] = '1'
# because if you specify a different target arch,
# you will need this to build v8 snapshot

You should now be able to use –with-snapshot for the cross-compile.

Note: Cross-compiling with snapshot only works on ia32 builds for the moment. I will post an update so cross-compiled will work on x64 when I figure out the problem.

Extra libraries

I ended up needing to point the build to another library location for libz. This could apply if you need to supply supplemental library directories for your hardware. This a target libz library.

parser.add_option("--libz-path",
    action="store",
    dest="libz_path",
    help="libz location. default uses standard location.")

and

def configure_libz(o):
  o['libraries'] += ['-lz']
  if options.libz_path:                               # add these two lines
    o['libraries'] += [' -L%s' % options.libz_path]   #

This will pass down to the v8.gyp during configuration, and should force a build of a host mksnapshot if you specify a –dest-cpu. On your target, you will notice a faster startup of node.

Cross-compile V8 for ARM

So you want your own little javascript framework for your favorite ARM device? Us too… OK, here we go.

Getting V8 to compile is not a trivial issue because V8 actually writes its own assembly during operation. Remember, V8 at its core is a JIT compiler for Javascript.

ARM arch? Figure out what you have.

Chances are you are cross-compiling V8 with your target as ARM. If you are, you need to find out what kind of hardware you have; specifically what type of floating point your hardware can do and do you support VFP and VFP3. If you already have Linux up on your hardware, /proc will reveal some good information. Check out the Features line. If you don’t see vfpv3 you don’t want the vfp3 options for V8. And if you don’t see vfp and neon, then you want armeabi=soft (not even softfp, which is using hardware instructions).

root@freescale /proc$ cat /proc/cpuinfo
Processor : ARM926EJ-S rev 5 (v5l)
BogoMIPS : 226.09
Features : swp half thumb fastmult edsp java      NOTE: MISSING: 'vfp'
CPU implementer : 0x41
CPU architecture: 5TEJ
CPU variant : 0x0
CPU part : 0x926
CPU revision : 5

Hardware : Freescale MX28EVK board
Revision : 0000
Serial : 0000000000000000

Without Snapshot

First, get V8 cross compiling without snaphost enabled. Snapshots enable faster start of a V8 instance since it will have a pre-compiled set of standard data structures and functions. But its not necessary for functionality.

x86_64: If running uname -f gives you x86_64, your are on 64-bit linux. You need to switch to a 32-bit install to get this to cross compile right now. This is a limitation with V8′s code with tests to make sure you are cross compiling from ia32. I’ll update this if I can figure out how to cross compile on x64.

Here is a cross-compile script I used:

#!/bin/sh
CROSS_BASE=/opt/freescale/usr/local/gcc-4.4.4-glibc-2.11.1-multilib-1.0/arm-fsl-linux-gnueabi
PREFIX_BIN=$CROSS_BASE/bin/arm-fsl-linux-gnueabi

export TOOL_PREFIX=$PREFIX_BIN
export CXX=$TOOL_PREFIX-g++
export AR=$TOOL_PREFIX-ar
export RANLIB=$TOOL_PREFIX-ranlib
export CC=$TOOL_PREFIX-gcc
export LD=$TOOL_PREFIX-ld

export CCFLAGS="-march=armv5te -mtune=arm926ej-s -mno-thumb-interwork"
# -march=armv5te -mtune=arm926ej-s -mfloat-abi=softfp

#export ARM_TARGET_LIB=$CTOOLS_LIB
scons wordsize=32 snapshot=off arch=arm vfp3=off armeabi=soft sample=shell

I did not need to use the ARM_TARGET_LIB var, and frankly could not find anywhere where it was used in the compile. For more information, see Google’s cross-compiling guide for V8. On successful build, you should have an executable called shell. Since we chose not to build V8 as a shared library for now, you can just copy this executable over to the dev hardware and test.

Step by Step

Run the executable…

root@freescale ~$ ./shell
V8 version 3.8.7.1 [sample shell]
> var num=3.3;
> print(num*3.3);
10.889999999999999
> quit();

If it fails, go back turn most options down/off. Then enable them one at time in an effort to find which feature is breaking the execution. Stuff to consider:

  • GCC switches: -march=XXX, -mtune=XXX, -mno-thumb-interwork
  • V8 scons options: vfp3=off/on, armeabi=off/on
  • V8 scons snapshot=off  - Later try to move to snapshot=on

Also, make sure you do a scons –clean (that’s a dash-dash) to cleanup objects when rebuilding.

Errors?

root@freescale ~$ ./shell
Illegal instruction

My first builds didn’t work right as I did not have the floating point switches set correctly. An illegal instruction is likely caused by some assembly that your processor does not support. But let’s use gdb (if your dev hardware has it) to quickly find out.

root@freescal ~$ gdb ./shell
(gdb) layout split
(gdb) r
(gdb) bt
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
 x x
 x x
 x x
 x [ No Source Available ] x
 x x
 x x
 mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
 >x0x78680 <_ZN2v88internal4HeapC1Ev+216> vldr d7, [pc, #424] ; 0x78830 <_x
 x0x78684 <_ZN2v88internal4HeapC1Ev+220> mov r3, #5242880 ; 0x500000 x
 x0x78688 <_ZN2v88internal4HeapC1Ev+224> add r8, r4, #1392 ; 0x570 x
 x0x7868c <_ZN2v88internal4HeapC1Ev+228> add r8, r8, #4 ; 0x4 x
 x0x78690 <_ZN2v88internal4HeapC1Ev+232> str r3, [r4, #392] x
 x0x78694 <_ZN2v88internal4HeapC1Ev+236> mvn r3, #-2147483648 ; 0xx
 mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
multi-thre Thread 1073867 In: v8::internal::Heap::Heap Line: ?? PC: 0x78680
#1 0x000b26fc in v8::internal::Isolate::Isolate ()
#2 0x000b2c10 in v8::internal::Isolate::EnsureDefaultIsolate ()
#3 0x000b2c8c in global constructors keyed to _ZN2v88internal8ThreadId18highest
_thread_id_E ()
#4 0x002831c4 in __libc_csu_init ()
#5 0x401bf430 in __libc_start_main () from /lib/libc.so.6
#6 0x0000a254 in _start ()
(gdb)

The top assembly instruction is the culprit. vldr is an ARM assembly instruction for floating point. My hardware does not support this at all, so it needs complete soft-float, which is the armeabi=soft option handed to scons for V8. You can do a similar test if necessary to determine the cause of the crash.

Snapshots

Snapshots will make initial startup of V8′s instance faster. You will notice you can simply add snapshot=on to you scons command. But doing that in your cross-compile script will results in:

...
obj/release/version.o obj/release/zone.o obj/release/snapshot-empty.o -lpthread
obj/release/mksnapshot obj/release/snapshot.cc --logfile "/home/ed/v8/v8-src/obj/release/snapshot.log" --log-snapshot-positions
/bin/bash: obj/release/mksnapshot: cannot execute binary file
scons: *** [obj/release/snapshot.cc] Error 126
scons: building terminated because of errors.

This is because mksnapshot is a ARM binary, and that’s not what we need. Instead you need to build V8 once with an ARM simulator which will generate ARM code, with a x86 binary. Then this will be used to generate the snapshot to build the ARM binary with snapshot enabled.

Below is a full script which does both. This is built off Google’s information located here, but with a few minor changes.

Notice that CCFLAGS now has the -lstdc++ library tacked on. For some reason, during this two stage build, with two different build directories, libstdc++ was not being linked in. Not sure why, and your mileage will likely vary. My result was an error such as:

... undefined reference to `__cxa_pure_virtual'

Which indicates you don’t have the libstdc++ lib. Also, make sure you set the vpf3= and armeabi= options correctly when building the simulator as well, or otherwise your startup will see an Illegal Instruction again. Without snapshot the startup code is compiled when V8 is run on the target, and with snapshots its pre-stored from simulator’s output. So those options handed to scons for the simulator must be essentially the same as when cross-compiling with out snapshots.

The full script:

#!/bin/bash
V8DIR=..
function print_usage() {
    echo "$0 [ with-snapshot ]"
    exit
}
# some ugly option processing...
if [ $# -gt 1 ]; then
    print_usage
fi
if [ $# -gt 0 ]; then
    echo "Here"
    if [ "$1" == "with-snapshot" ]; then
	SNAPSHOT="1"
    else
	print_usage
    fi
fi

if [ ! -z $SNAPSHOT ]; then
    if [ ! -d "host" ]; then
	echo "Making ./host"
	mkdir host
    fi
    if [ ! -d "target" ]; then
	echo "Making ./target"
	mkdir target

    fi
    echo "Building simulator..."
    cd host
    scons -Y$V8DIR simulator=arm vfp3=off armeabi=soft snapshot=on
    mv obj/release/snapshot.cc $V8DIR/src/snapshot.cc
    cd ..
fi

CROSS_BASE=/opt/freescale/usr/local/gcc-4.4.4-glibc-2.11.1-multilib-1.0/arm-fsl-linux-gnueabi
PREFIX_BIN=$CROSS_BASE/bin/arm-fsl-linux-gnueabi

CSTOOLS_LIB=/opt/ltib/rootfs/lib
CSTOOLS_USR_LIB=/opt/ltib/rootfs/usr/lib

export TOOL_PREFIX=$PREFIX_BIN
export CXX=$TOOL_PREFIX-g++
export AR=$TOOL_PREFIX-ar
export RANLIB=$TOOL_PREFIX-ranlib
export CC=$TOOL_PREFIX-gcc
export LD=$TOOL_PREFIX-ld

export CCFLAGS="-march=armv5te -mtune=arm926ej-s -mno-thumb-interwork -lstdc++"
# -march=armv5te -mtune=arm926ej-s -mfloat-abi=softfp

export ARM_TARGET_LIB=$CTOOLS_LIB

if [ ! -z $SNAPSHOT ]; then
    cd target
    echo "Building for target..."
    scons -Y$V8DIR wordsize=32 snapshot=nobuild arch=arm vfp3=off armeabi=soft sample=shell
    rm $V8DIR/src/snapshot.cc
    cd ..
else
    scons wordsize=32 snapshot=off arch=arm vfp3=off armeabi=soft sample=shell
fi

Run as

./setup-cross.sh with-snapshot

for building with a snapshot. Your finished executable is in ./target