Honest to a Segfault

My latest experiment is "slide casting" — here's Contributing to SpiderMonkey (a slidecast that's less than four minutes long):

Links

• "New to SpiderMonkey" wiki page

• JS Engine: good first bugs

• Hacking the JS shell screencast

• Creating JavaScript tests

• ChatZilla extension

• Building Firefox

• Mozilla manifesto (English)

• Release cycle

• ECMAScript specification

Transcript

Contributing to SpiderMonkey

This is a short presentation about contributing to Mozilla's SpiderMonkey JavaScript engine.

Business

As a guy who writes code, there are a few basic things I ask before I jump into any project:

• Will I learn?

• Do the people rock?

• Will it ship?

• Does it matter?

I guarantee that you'll learn from working on SpiderMonkey. It's an important language implementation created by some of the most brilliant and approachable coders I've ever had the privilege of working with.

We ship like clockwork. When a patch gets submitted to trunk, it's in the Firefox release 18 weeks later.

Hack: this technology could fall into the right hands [image]

And when it comes to finding meaning in your work, Mozilla makes life easy.

In my opinion, the Mozilla mission is technological utopianism at its finest. If you believe in using your technological superpowers to help people, Mozilla is for you.

Wrench

If you know how to write and debug C++ code, you have the skills to hack SpiderMonkey. We don't go crazy with the C++, so C coders should also feel pretty confident. The only tools required are the ones necessary to build Firefox.

SpiderMonkey is a language implementation, but don't let that get to you. Once you get your hands dirty (say, by fixing a few minor bugs) you'll realize that language VMs are no different from the other systems-level programs that you know and love.

See

The Mozilla project coordinates effort through Bugzilla. Every bit of work that we intend to do on the engine is tracked as a bug under the "JavaScript Engine" component at bugzilla.mozilla.org.

The JS team tries to tag good first bugs. If you see a good first bug that interests you, feel free to go in and make a comment stating your interest.

If you'd like to ease into development a little more, you can check out the latest ECMAScript specification and use that to create tests for our test suite. This is a great way to ensure SpiderMonkey engine quality and cross-browser compatibility.

Do

In typical open-source style, once you've found something that interests you, hack away!

And feel free to sample from the buffet: every bug that you work on teaches you about a different aspect of the engine.

You may also stumble onto a part of the engine that you'd like to specialize in — we loving having domain experts hacking on our code as well!

Code

Once you've made a working improvement to the engine, make sure you get your work in! Add your changes as an attachment to an existing bug, or create a new bug in the JavaScript Engine component.

When you improve the engine, you can get your name added to about:credits, in a product that ships to something like half a billion users, which I think is pretty cool.

Lots of great details and walkthroughs are available on the "New to SpiderMonkey" wiki page.

Barrel (#coding, #jsapi)

Friendly people hang around in these IRC channels at irc.mozilla.org. #coding is for general questions, whereas #jsapi is for JS engine internals discussion. You can easily install ChatZilla as a Firefox add-on to get on IRC.

If you've had bad experiences with IRC in the past, fear not!@ I know, from personal experience, that the IRC moderators in these channels have a zero-tolerance policy for disrespectful behavior. We love (and are) our community.

On my back

I haven't provided any kind of engine internals overview, but I think this may be just enough information to get you intrepid hackers going.

I may find time to do more screencasts in the future, but don't wait on me. (I'm new to this whole medium and prefer to write code. ;-) In the meantime, there's a screencast intro on hacking the SpiderMonkey shell available on my blog.

Around

The beauty of software, especially open source, is that you can mess around without taking any risks and by satisfying very few dependencies (i.e. a computer and the ability to install open source software).

Like the slogan says, with you hacking on Mozilla, the technology may have feallen into the right hands.

So, I hope that you'll consider hacking with us!

Note

Please excuse my use of the colloqualism, "as a guy who writes code." On a second listen I realize it may be poorly placed, because I'm using my own criteria as an example of an arbitrary person who might be considering contributing to the Mozilla project — no gender implication for contributors was at all intended!

More fortunately, this note is a great opportunity for me to plug WoMoz, Mozilla's project to promote women's involvement in open source and encourage contributions. You can find members on #womoz on irc.mozilla.org.

Thoughts on the idea of "slidecasts"

Just to get this established up front: I'm super rusty at presenting any kind of material. Also, I've never tried to record a presentation on the same computer that I was reading notes off of. (Case in point: you can hear the clicking of a keypress when I change slides.)

Despite all this hedging, I'm not sure about slidecasts as a medium. I sometimes fumble when I ad-lib, so I effectively had to write out a whole script for these slides. That's why it sounds like I'm reading off of a piece of paper.

Screencasts (as opposed to slidecasts) are different because you're walking through a sequence of on-screen editing actions that are inherently difficult to put into words. It's also a lot of fun to see how somebody else uses their development environment.

Slidecasts harness the poignant messaging of slides, but lose the body language of recorded audience presentations, which is clearly an important component. Turning the slidecast script into words would have been simple, and potentially more accessible to people who don't have the time to watch video content at all.

...or maybe it's humanizing? I'm not sure. Perhaps I have to add more soaring rhetoric and fancy slides to make spoken word worthwhile.

Clearly, more experimentation is needed!

I saw Gary Bernhardt's DestroyAllSoftware screencasts last night and thought, "Wow, that's a great idea! I bet it takes a lot less time to make a screencast than to write a blog entry with thousands of gross words in it."

And here you have it: my first screencast. Since it's only five minutes long, I was able to make it right after breakfast! It shows how to get started building the JS shell and how to add a new shell builtin that performs very important functionality.

I'm not very good at speaking and typing concurrently, so be gentle! (Also, it's hard to see if it's not fullscreen and HD quality, so I recommend using that.)

Update 2011-10-19 1700: @dukeleto points out that, if you need to work around recent issues with Mercurial clone timeouts on Mozilla servers, you can also visit the Mercurial web view for the tip revision and download via the bz2 link displayed at the top.

Casting a pointer (like Foo *) to a reference (like Foo &) via reinterpret_cast or a C-style cast probably doesn't do what you want.

References ("refs") exist so that you can make libraries with user-defined constructs that "feel like" a built-in language abstraction. Refs are definitely confusing if you've transitioned from C to C++ — they're "pointerish" in the sense that the compiler ultimately boils them down to pointer values, but "not" in the sense that the language semantics restrict their use. [*]

I came across one such casting bug today, and wondered what the compiler actually emits for it.

As it turns out, GCC warns when you cast a pointer to its corresponding ref type:

test.cpp:12:23: warning: casting ‘int*’ to ‘int&’ does not dereference pointer

Unfortunately, if you cast it to a corresponding const ref type it stays silent. Consider this snippet of C++ code:

#include <stdio.h>

extern int SomeGlobal;

void DumpValue(const int &value)
{
    printf("%d\n", value);
}

int main() {
    int *pval = &SomeGlobal;
    DumpValue((const int &) pval);
    return 0;
}

Note that the correct approach is to use the deref operator (*) on pval to turn it into an int &, which is compatible with the const int & signature of DumpValue.

After a quick give-me-the-assembly command line sequence:

g++ -o test.o -c test.cpp
objdump -d -r test.o # Get assembly with inline linker relocation directives.

We can see the resulting x64 assembly:

0000000000000025 <main>:
  25:   55                      push   %rbp
  26:   48 89 e5                mov    %rsp,%rbp
  29:   48 83 ec 10             sub    $0x10,%rsp
  2d:   48 c7 45 f8 00 00 00    movq   $0x0,-0x8(%rbp)
  34:   00
                        31: R_X86_64_32S    SomeGlobal
  35:   48 8d 45 f8             lea    -0x8(%rbp),%rax
  39:   48 89 c7                mov    %rax,%rdi
  3c:   e8 00 00 00 00          callq  41 <main+0x1c>
                        3d: R_X86_64_PC32   _Z9DumpValueRi-0x4
  41:   b8 00 00 00 00          mov    $0x0,%eax
  46:   c9                      leaveq
  47:   c3                      retq

Walking through it step by step:

• Instruction 2d is placing the address of SomeGlobal into the stack frame, at location -0x8(%rbp). [†] It currently has $0x0 as a value, with a note for the linker to replace that with the address for SomeGlobal when the linking process figures out where SomeGlobal lives.

• Instruction 35 computes the address of that stack slot with a lea instruction (which is like a fancy-pants add).

• Instructions 35 and 39 make that address of the stack slot into the first argument (%rdi) to DumpValue.

So, the argument won't contain the address of SomeGlobal, like we were hoping to provide to DumpValue, but the stack slot address instead. [‡] The cast resulted in a pointer to its operand — the behavior that you would expect if you took a value type and casted it to a ref, like so:

#include <stdio.h>

struct MyStruct {
    int foo, bar;
};

void DumpValues(const MyStruct &ms)
{
    printf("%d %d\n", ms.foo, ms.bar);
}

int main(void) {
    MyStruct ms = {42, 1024};
    DumpValues(reinterpret_cast<const MyStruct &>(ms));
    return 0;
}

Footnotes

The aroma is calling you. The best part of waking up is x86-64 in your CPU. [*]

Let's say it's early in the morning and you're a little cranky, but you want to see if/how compiler converts the following into branch-less code:

extern int a, b, c;

void test() {
    c += a == b;
}

You're using the boolean result from the binary comparison operator to, potentially, bump the value in c. You know that the result of the equality is a bit value because the spec wouldn't lie to you — you've been through so much together:

Each of the operators yields 1 if the specified relation is true and 0 if it is false. The result has type int.

—ISO C99 6.5.9 Equality Operators (3)

By using extern variables as operands, you prevent the compiler from constant-folding or optimizing anything away, since it doesn't know squat about the variables except for their type.

To check out what the compiler generates, all that you have to run is the following:

gcc -O3 -Wall -c foo.c
objdump -d -r -Mintel foo.o

To get a disassembly of the optimized instruction sequence in Intel assembly syntax, with relocations — extern placeholders whose actual memory addresses get filled in by the linker — displayed inline:

0000000000000000 <test>:
   0:   8b 05 00 00 00 00       mov    eax,DWORD PTR [rip+0x0]        # 6 <test+0x6>
                        2: R_X86_64_PC32        a-0x4
   6:   3b 05 00 00 00 00       cmp    eax,DWORD PTR [rip+0x0]        # c <test+0xc>
                        8: R_X86_64_PC32        b-0x4
   c:   0f 94 c0                sete   al
   f:   0f b6 c0                movzx  eax,al
  12:   01 05 00 00 00 00       add    DWORD PTR [rip+0x0],eax        # 18 <test+0x18>
                        14: R_X86_64_PC32       c-0x4
  18:   c3                      ret

The crux of the fun is the sete opcode, part of the set* family of 8-bit opcodes that set the 8-bit operand to the bit value of a condition flag.

Then, because 8-bit opcodes preserve the higher bits of the register they operate on, and you need to perform a clean add of a bit value (with no junk left hanging around in the higher bits), you zero-extend the 8-bit value into the corresponding 32-bit form.

Finally, you have the bit value in eax which you can simply add to (the placeholder for) c.

It's also fun to note that, even if you used a 64-bit wide type (a long on an LP64 system like mine), the same sign-extending code sequence would be generated! Because 32-bit operations don't preserve the higher (32) bits of the register they operate on, but instead clear them out, the movzx instruction actually zeroes out all the bits in rax aside from the 8 in al that you're zero extending. For even more tutorial-imbuing goodness, you can try switching the extern declaration over to long and test it out for yourself.

Footnotes

I am totally taken aback by the lack of hyperbolic romanticism in the foreword of the programming language book that I just got. There are horribly boring-sounding hooks along the lines of "computers are ubiquitous in the 21st century" and "connecting the theoretical foundations of computer science to modern platform architectures". Hopefully people don't judge a book by its cover or its foreword.

Pragmatic rhetoric is uninspiring — you could just as easily write a foreword for a book on ripping up floorboards and talk about how "floorboards are ubiquitous in the 21st century" and "connects the theoretical foundations of wood science to modern house architectures".

If I were to write a foreword that mentioned the reasons you should be interested in programming languages, it would go roughly like this, to which I hope there is no analogy for ripping up floorboards:

Foreword

Rejoice, programming languages are the irrigation ditches of your mind-goo!

You've got brilliant ideas brewing inside your head, trying to claw their way out of your little mind and escape into the world. Unfortunately, the meat shell that your ideas live in is quite limited — you can barely eat and talk at the same time, and even then, you're not supposed to be talking with your mouth full. It's bad manners.

Luckily for you, computers provide another venue to express and execute your abstract thoughts. The way you express yourself to these unquestioning harbingers of awesome is through programming languages, whose programs cause actions to be taken in the computing device's virtual world.

Much like ice cream, programming languages come in many flavors (and some have those great cookie dough chunks that you find when you first bite into them). The flavor of a programming language shapes the way that people express and reason about their abstract thoughts — a result of the language's unique design and implementation. Because there's no "best" way to channel your thoughts into a computing device, our work on programming languages is never done!

What [whoever] has written here is [probably] a wonderful explanation of the way we currently bridge the gap between the mind and the computing device for the flavors of programming language we've invented-and-used thus far. Learning from historical successes and pitfalls is key to really understanding existing programming languages and evaluating the design decisions that you'll be making for the programming languages of tomorrow.