Abhijith Chatra

Inline Cache

2015-12-29T00:00:00+00:00

In this post, we are going to understand the inline cache with respect to Chakra. We will briefly look at the need for inline cache as well. If you haven’t heard of inline cache at all, this wiki page and the type blog are necessary to read before we deep dive. To quote from wiki: The concept of inline caching is based on the empirical observation that the objects that occur at a particular call site are often of the same type. In those cases, performance can be increased greatly by storing the result of a method lookup “inline”, i.e. directly at the call site.

Note, in this context of blog call site means any property access location (line of code), not just a location where a function is called. First let us understand the cost of property lookup in Chakra.

Blog in progress…

###Cost of property lookup

Example is the best way to begin: ```js function Car(make, model) { this.make = make; this.model = model; }

var mycar = new Car(“Honda”, “Accord”);

function GetModel(machine) { return machine.model; } print(GetModel(mycar)); ```

Function GetModel returns model property from machine object. Typically machine is instance of Car. Though GetModel can be invoked from a different type of object such as SewingMachine and still the contract of code is valid. Sewing machines also have model property.

function SewingMachine(brand, model) {
  this.brand = brand;
  this.model = model;
}
var mymachine = new SewingMachine("Brother", "CS6000i");
print(GetModel(mymachine));

When the model property is accessed in script following steps are taken. (Read about type here)

Fetch the type (machine->type) of the object.
Invoke dictionary lookup in property map ((machine->type->typeHandler) to get the slot number corresponding to property model. Lets say property model is at slot 1;machine->slots[1]

Step 2 is super expensive for runtimes.

###Dumping bytecode

Let us dump the bytecode using debug version of ch.exe to see inline caches. Ch.exe is a lightweight console host for hosting ChakraCore. See using ChakraCore section here for how to build ch.exe .

ch.exe test.js -dump:bytecode

Function GetModel ( (#1.2), #3) (In0, In1) (size: 11 [10])
      5 locals (1 temps from R4), 2 inline cache
  Line  12: return myCar.model;
  Col    6: ^
    0012   ProfiledLdFld        R0 = R3.model #0 <0>
    0016   Br                   x:0021 (   8)

Type

2015-12-22T00:00:00+00:00

Javascript is an untyped language. Type in the context of this blog refers to a data structure in the Chakra runtime, which holds additional information about an object. Type is also popularly known as hidden class. Type shares data about a class of objects. The sole purpose of a type in the runtime is to enhance the performance of the user script code. If you are building your own javascript runtime and didn’t care about performance or the memory, you don’t need type.

Chakra has two kinds of type, static type and dynamic type. The static type is for the primitive object which can’t store properties in it. For example: String (“hello”), Boolean (true or false), Number (10.4) etc. The [Dynamic object] (https://github.com/Microsoft/ChakraCore/blob/master/lib/Runtime/Types/DynamicObject.h#L46) which can store properties have dynamic type. Example: {} or new Point(). Let us go a bit deeper into static type and dynamic type and see how they share.

###Static Type Every RecyclableObject holds a pointer to a type. Every object class in Chakra inherits from RecyclableObject except tagged floats. Let’s take an example:

var greeting = "hello";
var message = "open source";

Here we have two literal strings greeting and message which Chakra tracks as a JavascriptString. JavascriptString class inherits from RecyclableObject. Both greeting and message point to a shared type.

This type has the typeId set to TypeIds_String and shared between two literal string objects. It may seem an overkill to share an integer typeId when you have to add a pointer to that type in the literal string objects. In reality, type contains a lot of additional data, not just the typeId. Following is the field dump of the type. Prototype and entrypoint are shared as well.

C++ Js::Type typeId TypeIds_String (7) Js::TypeId flags javascriptLibrary* prototype* entryPoint* propertyCache*

JavscriptString type is a static type. Dynamic objects provide a lot more opportunity to share information. Again dynamic objects have a dynamic type.

###Dynamic Type Let us take an example first:

function Point(x, y)
{ 
  this.x = x;
  this.y = y;
}
var one = new Point(10,20);
var two = new Point(40,50);

print(one.x);
print(two.x);

Above script creates two Point objects which have properties x and y. Chakra needs to store following information in the runtime:

Objects one and two have properties x and y
Object one has property x whose value is 10 and property y whose value is 20
Object two has property x whose value is 40 and property y whose value is 50

Above information needs to be retrieved when the script wants to get or set the property. Note (1) is shareable among multiple objects. (2) & (3) are specific to individual dynamic objects. A typical way to store this is by building a property map and a slot array.

Property map, maps between a property and a slot number. Example: Property x is present at slot 0.
Slot array stores the values. Example slots[0] contains the value of property x.

Property map is stored in a dynamic type (to be specific, in a type handler which is part of dynamic type). When script does one.x, Chakra fetches property map in the one->type to figure out the slot number corresponding to x which is 0 here. Then it fetches one->slots[0] to get the value 10.

The following diagram illustrates the same.

A single type can be shared by all the objects created from the constructor Point. Even if you create a million Points, you just need a type to represent them all. Memory saving by type is just a tip of the iceberg. Tons of optimization are possible by the concept of type. I will note a bunch here:

Inline Cache
Object type specialization in JIT
Function specialization

Going deeper into these optimizations necessitates a separate post. First we need to understand how this type is built. Typehanlder which is part of a dynamic type handles how the property map is stored and how a brand vanilla dynamic object gets its type.

###Typehandler

Typehandler in a dynamic type serves two important goals:

Maintains a property map which maps between a property and a slot number.
Maintain successor types.

We have already seen that property map is to share between objects. But what is a successor type? Let us go back to an example:

var one = {};       
var two = {};       
//Objects one and two points to Type1

one.x = 10;     
//Object one points to Type2 and object two points to Type1
starwars1(one, two);

one.y = 20;     
//Object one points to Type3 and object two points to Type1
starwars2(one, two);

two.x = 40;    
//Object one points to Type3 and object two points to Type2
starwars3(one, two);

two.y = 50;  
//Object one and two points to Type3
starwars4(one, two);

The assumption here is functions starwars doesn’t mutate objects, it just accesses properties off them. It is optimal for one and two to share the exact same type when they are constructed. Both these objects end up with property x and y as well. So they can share the same type when the function starwars4 is invoked. In between these two states, we can see that objects one and two differ in what they contain. At the call site of starwars2 object one has x & y but object two has none at all. Obviously, they can’t have the same type. How do we make sure all the objects with same properties end up with the exact type if they are constructed at different point of time?

Successor type in typehanlder comes to the rescue. Each typehandler holds a successor type which tracks what new type to assign to an object if it gets a new property. This process is known as type promotion or type transition. Following diagram tries to illustrate the same.

When a property x is added to object with Type1, next type of the object is going to be Type2. Objects with just property x will share Type1.

As script adds properties to objects, type promotion occurs quickly as typehandler holds a pointer to the next type. The successor type is present to improve the speed of type promotion. One can build a type system without the successor type. Though for objects to share types, you will have to search through an entire set of types to figure out what is the next type your object needs to get when you set a new property.

To understand this in Chakra codebase take a look at SimpleTypeHandler. SimpleTypeHanlder has a property map named typePath in the base class which has a [tiny dictionary] (https://github.com/Microsoft/ChakraCore/blob/master/lib/Runtime/Types/TypePath.h#L73) to map a propertyId (synonymous to property name) to a slot number. It also consists of successorTypeWeakRef which holds the pointer to next type. A variant of SimpleTypeHandler known as [PathTypeHandler] (https://github.com/Microsoft/ChakraCore/blob/master/lib/Runtime/Types/PathTypeHandler.h#L243) holds map known as PropertySuccessorsMap to multiple successors.

###Summary

We looked at why objects have a type and how it shares data. We briefly looked at why typehandler stores a pointer to successor type. Note there are various other typehandlers which share data differently or not share at all. All have their own use cases for performance or memory.

In my opinion biggest advantage of type is inline cache. We will look into the inline cache in the next post.

Hope you liked this post and please post any feedback in the comments.

Tagged Float

2015-12-15T00:00:00+00:00

A [tagged pointer] (https://en.wikipedia.org/wiki/Tagged_pointer) is a well know concept which every virtual machine (VM) tries to exploit. Unlike some other VM’s Chakra doesn’t tag a pointer. Instead, Chakra tag’s the non-pointer a.k.a a float or an int. For the purpose of this blog, I will illustrate the implementation of tagged float in 64 bit. In the context of this blog float means double-precision 64-bit format IEEE 754-2008 as specified by ECMA262. Chakra doesn’t tag floats in 32 bit but tags integers. On 64 bit Chakra tags both floats and integers. First let us see our object representation, it’s size and the need for tagged floats.

###Object representation Javascript is a Garbage Collected (GC) language. Any object or primitive (which represents javascript var) is accessed as a void pointer named Var in Chakra runtime.

C++ typedef void * Var;

Var typically points to a RecyclableObject. RecyclableObject is the root of the object hierarchy which all other objects inherit. This necessitates a vTable pointer which consumes 8 bytes for each object. It holds an additional pointer to Type which accounts for another 8 bytes. Type structure disambiguates between various kinds of RecyclableObjects such as strings, numbers, dynamic objects and can be shared between multiple objects. Every RecyclableObject has following two fields:

C++ __vfptr* // 8 bytes type* // 8 bytes

In a nutshell, 16 bytes are required to represent a simple object (again in x64). Now let us take an example. Following speed javascript variable holds a float.

var speed = 10.4;

To represent this Var in the engine, Chakra need to create an object named JavascriptNumber which inherits from RecyclableObject and can store a float value (10.4). Total bytes required is 24 (sizeof(Js::JavascriptNumber) == sizeof(Js::RecyclableObject) + sizeof(double)). Turns out our GC allocates at 16 byte boundary. 24 bytes is rounded off to 32 bytes. Chakra needs 32 bytes to represent a JavascriptNumber. In addition, to this 8 byte Var pointer is necessary for the runtime to point to this object. Can we avoid this overhead for every var pointing to a float? If we could encode a float in a pointer and mark it as a non-pointer, the runtime can recognize it and we save on bytes.

###Extra bits in a pointer Let us look at memory address allocated by the GC carefully. Couple of characteristics of pointers:

Bottom 4 bits are always going to be zero. (Remember our GC allocates at 16 byte boundary)
Top 16 bits are going to zero as the operating system only uses bottom 48 bits to represent virtual memory address (256TB is good enough).

We can party with these extra bits. One can use these always zero bits to encode a float. But how will the runtime differentiate between a valid pointer or a float encoded in a pointer? Can we encode the entire float which is 64 bit inside a pointer which is also 64 bit and don’t lose precision? First it is important to understand the floating point representation.

###IEEE 754 floating point representation Now let us look at the 64-bit double precision floating point value:

Sign	Exponent	Fraction
1 [63]	11 [62-52]	52 [51-00]

See [this] (http://steve.hollasch.net/cgindex/coding/ieeefloat.html) blog for more information on floating-point format. The interesting part is the exponent. If all the 11 bits in the exponent are 1 it can represent 3 values:

Positive infinity.
Negative infinity.
NaN

NaN’s are represented by a bit pattern with an exponent of all 1’s and a non-zero fraction. If the fraction is all zero it can be either +Infinity or -Infinity depending on sign bit. This lets tons of ways of expressing NaN. Ecma262 specifies that all NaN’s are treated equally. So we just need one. Chakra canonicalize’s all the NaN’s to just the one shown below which is, in fact, a QNaN.

C++ static const uint64 k_Nan = 0xFFF8000000000000ull; static const uint64 k_PosInf = 0x7FF0000000000000ull; static const uint64 k_NegInf = 0xFFF0000000000000ull;

Now let us dig deeper into our tagging scheme.

###Tagging scheme Remember our goal here is to pick a tagging scheme in a pointer so as to:

Easily differentiate between a pointer and a float.
Not lose any data while encoding a float.

From the above IEEE 754 floating representation we know that all float values (except NaN, +Infinity, -Infinity) are guaranteed to have at least one of the exponent bits not set. So we xor all floats with 0xFFFC0000 00000000 or 0xFFFC«48 and store them in the memory as pointers. This magic xor constant guarantees that all floats will have at least one bit set in the exponent part (bits 62-52). This magic constant also ensures that NaN, Infinity & -Infinity have 50th-bit set. To generalize all floats will have one of the top 16 bit set. Pointers won’t have any of the top 16 bit set.

A simple table to illustrate.

float value or pointer	Bit pattern in hex	Bit pattern after xor
0.0	0000000000000000	fffc000000000000
0.4	c02599999999999a	3fd999999999999a
Infinity	7ff0000000000000	800c000000000000
-Infinity	fff0000000000000	000c000000000000
NaN	fff8000000000000	0004000000000000
RecyclableObject*	00000209512b4e20	00000209512b4e20

Note: Chakra keeps RecyclableObject pointer values as is.

See links for floating point conversion calculator.

Runtime simply looks at top 16 bits (x >> 48 != 0). If any of top 16 bit is set, it is a float or else it is a valid pointer to a RecyclableObject. If it is a float, runtime gets the original float by xoring with 0xFFFC«48 (x === (x^(0xFFFC<<48)^(0xFFFC<<48)). Total memory spent on a float in the VM is just 8 bytes as Chakra directly stores the float inside a pointer.

To close this post Chakra does bit twiddling magic to save 32 bytes of memory for each var pointing to a float. Hope this helps. Please let me know the feedback either through email or leaving a comment here.

Hello open source

2015-12-14T00:00:00+00:00

I joined Chakra team back in 2009. I have hacked a bunch of code in Chakra engine. Eventhough I have spent a great deal of time in this codebase, I still have lot to learn. (I am not an expert in any sense).

Intention of this blog is to help new developers to make sense of basic building blocks in Chakra. Over the course I intend to cover all parts of the engine such as type system, jit, runtime etc. If you see any inaccuracy in my blog please point out.

Following is the list of articles. I will populate the list as I have more content. If you are particularly interested in a topic do let me know.

[Tagged float] (http://abchatra.github.io/TaggedFloat)
[Type] (http://abchatra.github.io/Type)

Thank you!