Introduction

Debugging Wasm? Put some whamm! on it!

whamm! is a tool for "Wasm Application Monitoring and Manipulation"1, a domain-specific language (DSL) inspired by Dtrace's D language. If you're building a new dynamic analysis for Wasm and are looking for a framework to support you, you're in the right place! This book will help you get on the right track when working with the language.

whamm! enables Wasm tool implementers to express their instrumentation using high-level abstractions of program events or locations at various levels of granularity, increasing the expressiveness, intuitiveness, and maintainability of the tool.

Here are some of the goals of whamm!:

  1. is high-level and intuitive
  2. instrumentation is easy-to-build, testable, and debuggable
  3. express instrumentation in terms of predicated probes
  4. can instrument events of different granularity
  5. provide behavior as Wasm functions, say where to call them in whamm!
  6. write instrumentation once, whamm! takes care of the injection strategy.

Let's take a moment to consider the scale of impact that this DSL could have on developer tooling by considering the following facts:

  1. WebAssembly is growing to use cases beyond the browser.
  2. Many languages can compile to Wasm.
  3. With whamm! write instrumentation once to support wide domain of apps.
    • Use engine instrumentation capabilities as available.
    • Use bytecode rewriting to support everything else.

This means that developer tools written in whamm! could support a vast domain of applications, making WebAssembly the future target platform for debugging.

Resources

Injection strategies:

Some helpful terms and concepts

  1. WebAssembly (Wasm): WebAssembly is a binary instruction format for a stack-based virtual machine. It is designed as a portable compilation target for programming languages.
  2. Instrumentation: When we say we are "instrumenting a program," at a high-level we mean we are "injecting some code into a program’s execution to do some operation." This definition is intentionally generic since instrumentation can really do anything we can imagine! You can use instrumentation to build debuggers, dynamic analyses, telemetry generators, and more.
  3. Dynamic analysis: A dynamic analysis is something that analyzes a program as it is executing (in contrast to a static analysis which analyzes a program that is not running). This type of analysis can gain useful insights into a program as it is able to access information that is not available statically (such as hot code locations, memory accesses over time, code coverage of test suites, etc.).
  4. Bytecode rewriting: This is an example strategy for injecting instrumentation logic into the application. It injects instrumentation through literally inserting new instructions into the application bytecode.

1: The 'h' is silent.

Getting Started

Here you will find information on how to begin writing instrumentation in the whamm! DSL.

Installation

The current way to install whamm! is to clone the repository, build the source yourself, and add the created binary to your PATH variable. In the future, users will be able to download pre-built binaries on the GH releases page as we have stable, tagged releases of whamm!.

Steps:

  1. Clone the whamm! repo
  2. Build the source code with cargo build
  3. Add the built binary to your PATH. This binary should be located at target/debug/whamm1.

Basic Test

A basic test you can run to make sure that the whamm! binary is on your path and working as-expected is running the following command: whamm --help. The CLI will provide information on various commands and options available for use.

Wasm monitors and manipulators

As mentioned in the introduction, whamm! can be used to either monitor OR manipulate a program's execution.

What we mean by monitor execution is collect some information about a program's dynamic behavior. This is commonly used for debugging, logging, and metric collection.

What we mean by manipulate execution is to literally change the program's dynamic behavior. Consider a specific feature of many debugger tools: using a debugger, a developer can set a breakpoint, inspect the current application state, and change the values of variables. This is an example of manipulating an application's dynamic behavior through changing the state and something we will support doing in whamm!.

Continue reading through this book's "getting started" content for how to write such monitors and manipulators.

Anatomy of Instrumentation in Whamm!

The core anatomy of instrumentation written in Whamm! consists of the Whamm! script (instr.mm) and the instrumentation library (instr.wasm).

Together, these files tell the Whamm! compiler where to insert instrumentation and what logic to inject at those points.

instr.mm

Where to insert instrumentation.

This Whamm! script describes a unit of instrumentation by specifying the points to probe in an application using the probe syntax. These probes contain the logic to inject at these match point in their bodies. This logic can call out to the user's instrumentation library, provided as lib.wasm.

lib.wasm

What logic to insert at the targeted application points.

It is optional to provide this library to Whamm!. It is only necessary if the user would prefer to write their instrumentation in a higher-level language or if the DSL does not have some syntax/tool necessary for the instrumentation logic. This language must compile to Wasm as the library must be provided to the Whamm! compiler as a Wasm file.

The user will need to keep in mind that the function signatures in their library that will be called in their probe bodies will need to have types compatible with the types currently provided by Whamm!. Otherwise, Whamm! will not be able to compile their instrumentation code.

Helpful Tools

Here are some tools that may help when working with Wasm:

  1. wabt, aka the WebAssembly Binary Toolkit
  2. wasm-tools

1: We recommend adding the binary built inside target/ to your path as this will enable you to pull the latest changes on master, build the latest version, and automatically have the latest binary on your PATH.

The Language

whamm! enables tool implementers to express their instrumentation in terms of program events and corresponding predicated actions; "When this event occurs during program execution, do these actions if this predicate (or conditional) evaluates to true." This abstraction provides a high-level and intuitive syntax that can target events at various granularities in the instrumented program.

Read on for an overview of the syntax and semantics of the language.

Language Concepts

  • Variables are used to store data.
  • Logical operations can be used to combine boolean expressions.
  • Ternary Expressions can be used for succinct conditional variable assignments.
  • Primitive types are numbers, booleans, and strings.
  • Various arithmetic operations can be used with numbers.
  • Strings are key for dealing with text, etc.
  • Tuples allow using multiple values where one value is expected.
  • Maps are key for storing large amounts of data, but they're implemented quite differently in whamm!.
  • Function definitions can be used to reuse code snippets.
  • Conditionals are if/else/elif statements used for simple control flow
  • Whamm! also provides helpful features to enable instrumentation expressiveness
    • Report variables are key for flushing the dynamic data collected by instrumentation during an application's run
    • Unshared variables are used to create an instance of a variable per probe match-site whose value is retained across site visits
    • Shared variables are used to create an instance of a variable that is shared by every probe match-site
  • And finally, probes are used to express instrumentation.
  • All of this syntax is used to write whamm! scripts.

Variables

Variables store data, such as numbers and strings.

// Declaring a new variable `<type> <var_name>;`:
i32 i;

// Assigning a value to a variable `<var_name> = <value>;`:
i = 0;

// Variables can also be set to the result of an expression `<var_name> = <expression>;`:
i = 1 + 2;
i = add(1, 2) + 9; // (assuming that the `add` fn is in scope and returns an `i32`)

Scopes

Each variable is associated with some scope, which is the range of the program in which it is active and accessible. We will see how there are scopes tied to functions, probes, and scripts. The syntax for declaring and assigning to variables is consistent across these contexts.

Logical Operators

Logical operators allow joining multiple boolean expressions. Like C/C++ and Java, the && and || operators provide for logical-and and logical-or. Both operators have short-circuit evaluation; they only evaluate the right-hand-side expression if the left-hand-side evaluates to true or false, respectively.

bool a;
a = false && false; // == false
a = false && true;  // == false
a = true && false;  // == false
a = true && true;   // == true

bool a;
a = false || false; // == false
a = false || true;  // == true
a = true || false;  // == true
a = true || true;   // == true

Ternary Expressions

whamm! supports a version of the "conditional" expression that chooses one of two values based on a condition. The syntax follows C, C++, and Java, which uses ? :.

// with declared types
int a;
a = 1 > 0 ? 16 : 27; // == 16
a = 1 < 0 ? 17 : 29; // == 29

Short-circuit evaluation

The ternary expression will only evaluate the branch corresponding to the value of the condition. In other words, it short-circuits.

Primitives

whamm! offers a small set of primitive types that are useful for performing arithmetic and representing data.

Booleans

With only two values, true, and false, booleans are represented in whamm! with the type bool.

bool x; // default == false
x = true;
x = false;

Integers

Right now, whamm! supports i32 (signed 32-bit values) and u32 (unsigned 32-bit values) integers, but will be supporting all numeric types provided by Wasm in the future.

// with declared types
i32 d; // default == 0
d = 0;
d = 9993;
d = -42;

The minimum decimal value for type i32 is -2147483648 (equal to -2^31) and the maximum value is 2147483647 (equal to 2^31 - 1).

The minimum decimal value for type u32 is 0 and the maximum value is 4294967295 (equal to 2^32 - 1).

Arithmetic

Arithmetic is fundamental to computation. whamm! defines a number of arithmetic operators on the primitive types.

Integer types

i32 a;
a = 0;
a++;       // increment == 1
a--;       // decrement == 0
a = 9 + 3; // add == 12
a = 8 - 2; // subtract == 6
a = 7 * 4; // multiply == 28
a = 9 / 2; // divide == 4
a = 5 % 3; // modulus == 2

Strings

NOTE: This functionality hasn't been fully implemented! More docs to come post-implementation!

Tuples

NOTE: This functionality hasn't been fully implemented! More docs to come post-implementation!

Maps

Whamm! provides maps for storage of key-value pairs. This is similar to java's Map and python's dict types. In fact, it is exactly Rust's HashMap type...since whamm! leverages this Rust type under-the-hood! If you are interested in how this works, see the Developer's Core Library documentation.

Instantiation

// No need to instantiate a map, it is automatically created as an empty map.
map<i32, i32> a;

Reading and writing elements

Reading and writing elements of maps uses the [ ... ] syntax like maps in many other languages.

map<i32, i32> a;
a[0] = 3; // map write
i32 b = a[0]; // map read

// maps can also contain tuples!
map<(i32, i32, i32), i32> c;
c[(0, 0, 0)] = 3; // map write
i32 b = c[(0, 0, 0)]; // map read

Bounds and null checks

Accesses of Whamm! maps are dynamically checked against the bounds.

map<i32, i32> a;
i32 b = a[0]; // produces Wasm trap through Rust

Functions

NOTE: This functionality hasn't been fully implemented! More docs to come post-implementation!

report Variables

The report keyword specifies variables whose values should be flushed. When monitoring an application execution, data describing observations should be stored in a report variable to make use of this flushing feature.

Using report is really shorthand for report unshared, see [unshared] variable documentation.

The default behavior of this "flush" is to print to the console (the core Whamm! library uses WASI to do this). Currently, this will be done on each write to at least one of the report variables; however, there are plans to make this configurable.

For example:

report i32 count;
wasm:opcode:call:before {
    // count the number of times the `call` opcode was used during the application's dynamic execution.
    // (a single global count)
    count++;
}

unshared Variables

If a variable is marked as unshared, an instance of this variable is made available to every match site for a probe. The scope of this variable is limited to the specific match site. The value of this variable will be stable on each entry into the probe's logic, meaning that it will not be reinitialized each time.

This can be used to collect data at a specific program point over time.

For example:

wasm:opcode:call:before {
    // collect the number of times each `call` opcode is executed during dynamic execution.
    // (as many counts as there are `call` opcodes in the program)
    // This variable will not be reinitialized each time this probe's body is executed,
    // rather, it will be the value it was the last time it ran!
    unshared i32 count;
    count++;
    
    // This variable will be reinitialized to 0 each time this probe's body is executed
    i32 local_variable;
    local_variable++;
}

shared Variables

NOTE: This functionality hasn't been fully implemented! More docs to come post-implementation!

If a variable is marked as shared, a single instance of this variable is available to every match site for a probe. The scope of this variable is limited to all match sites of a probe. The value of this variable will be stable on each entry into the probe's logic, meaning that it will not be reinitialized each time.

This can be used to collect data for some probed event over time.

For example:

wasm:opcode:call:before {
    // collect the number of times the `call` opcode is executed during dynamic execution.
    // (a single count tied to the wasm:opcode:call event in the program)
    // This variable will not be reinitialized each time this probe's body is executed,
    // rather, it will be the value it was the last time it ran!
    shared i32 count;
    count++;
    
    // This variable will be reinitialized to 0 each time this probe's body is executed
    i32 local_variable;
    local_variable++;
}

Probes

<probe_rule> / <predicate> / { <actions> }

We use the term probe to refer to this triple of probe_rule, predicate and actions.

When performing bytecode rewriting, whamm!:

  1. traverses the application's Wasm module to find the locations-of-interest as specified by each probe's probe_rule.
  2. checks if the predicate evaluates to false statically
    • if it does evaluate to false it continues on, not injecting the probe's actions
    • if it does not evaluate to false, it injects the probe's actions at that location along with the folded predicate.
      • if the predicate evaluates to true statically, it will simply inject the actions into the program un-predicated.
      • if the predicate does not fold to a simple boolean value, it will inject predicated actions into this location. The predicate will then be evaluated dynamically when the application runs to conditionally execute the probe actions.

Helpful info in CLI

whamm info --help

The info command provided by the CLI is a great resource to view what can be used as the probe match rule. This command provides documentation describing the match rule parts as well as the globals and functions in scope, which can help users learn about how to build their instrumentation.

The Probe Match Rule

provider:package:event:mode

The probe_rule is a way to express some "location" you want to instrument for your program. It is a hierarchical with respect to provider, package, event and `mode.

partdescription
providerThe name of the provider that supports this instrumentation capability used in this probe.
packageThe name of the package within the specified provider that supports the instrumentation capability used in this probe.
eventThe name of the event that would correlate with the location to insert this probe in the instrumented program.
modeThe name of the mode that should be used when emitting the probe actions at the event's location, such as before, after, and alt.

Each part of the probe_rule gradually increases in specificity until reaching the mode of your probe. Consider the following example match rule: wasm:opcode:br_if:before. This rule can be read as "Insert this probe before each of the br_if Wasm opcodes in my program."

Read through our instrumentable events documentation for what we currently support and our future goals.

The Predicate

/ <predicate> /

The predicate is a way to express some "conditional" you want to evaluate to true for the probe's actions to be executed. It further constrains the match rule. This aspect of a probe is optional to use. If there is no predicate for some probe, the actions will always execute when the probe's location is reached during program execution.

Constant Folding of Static Data

A probe can be predicated on both static AND dynamic data. To support this, when targeting bytecode rewriting, the whamm compiler performs constant propagation for statically-defined data. Meaning that since static information is known at compile time, the values of those variables will be substituted and expressions can be partially evaluated.

As an example, consider the following probe: wasm:opcode:br_if:before / pc == 25 && arg0 == 1 / {..}

This can be read as, attach a probe at the wasm opcode br_if, but only execute this logic if both the pc is 25 and the arg0 evaluates to 1. As we're traversing an application, we statically know when we're at the pc offset of 25 within a function. BUT we don't know the value of arg0 until the code is actually running! This is where partial evaluation comes in.

Let's say we're at pc 0 within some function, constant propagation and partial evaluation would look like this:

0 == 25 && arg0 == 1
false && arg0 == 1
false

The predicate evaluated to false statically! This means we should not attach the probe at this location, it's not a match!

Let's say we're at pc 25 within some function:

25 == 25 && arg0 == 1
true && arg0 == 1
arg0 == 1

Huh...we still have a part of the predicate left over! What should we do... We still need to predicate on this part of the expression, we just need to inject this partially-evaluated expression to run at runtime! So, we'd inject the probe body, wrapped with this condition!

Note that this does look a bit different for the Wizard target, whamm has to do some other tinkering with the predicate there, which enables the engine to run the correct part of the predicate at the correct time (match time vs. runtime).

The Actions

{ <actions> }

The actions are statements that are executed at the probe_rule's location if the predicate evaluates to true.

whamm! Scripts

Instrumentation (aka a monitor) is expressed as a set of predicated probes in a script with the .mm extension.

Here is a high-level view of the grammar for a whamm! script:

// Statements to initialize the global state of the instrumentation
global_statements;
...

// Function definitions to reuse code snippets
fn_name(fn_args) -> ret_val { fn_body; ... }
...

// An example of what a `probe` would look.
// There can be many of these in a monitor.
provider:package:event:mode / predicate / {
  probe_actions;
  ...
}

Instrumenting with the CLI

whamm instr --help

The instr command provided by the CLI enables developers to actually instrument programs.

Instrumentable Events

Currently available packages:

  • wasm:opcode, e.g. wasm:opcode:call:alt

Packages to be added:

  • thread operation events
  • gc operation events
  • function enter/exit/unwind events, e.g. wasm:fn:enter:before
  • memory access (read/write) events
  • table access (read/write) events
  • WASI component operation events, e.g. wasi:http:send_req:alt
  • BEGIN/END events
  • traps
  • exception throw/rethrow/catch events

Libraries

NOTE: This functionality hasn't been fully implemented! More docs to come post-implementation!

Testing Your Instrumentation

NOTE: This functionality hasn't been fully implemented! More docs to come post-implementation! This is a future research goal of ours.

Injection Strategies

Debugging and profiling programs are an integral part of engineering software. This is done through instrumenting the program under observation (inserting instructions that provide insight into dynamic execution).

The most-common instrumentation techniques, such as bytecode rewriting, inject instructions directly into the application code. While this method enables instrumentation to support any application domain, it intrudes on the program state space (possibly introducing bugs), complicates the implementation, limits the scope of observation, and cannot dynamically adapt to program behavior.

Instead, one can remedy these issues with bytecode rewriting by interfacing with a runtime engine that directly supports instrumentation. This technique can bring powerful capabilities into play as demonstrated by the Wizard research engine, in the ASPLOS paper Flexible Non-intrusive Dynamic Instrumentation for WebAssembly. This paper demonstrated how to build instrumentation support that protects the application-under-observation, provides consistency guarantees to enable composable tooling, applies JIT optimizations specific to instrumentation that make some tools run even faster than bytecode rewriting, and more. However, this technique is not as widely-used as bytecode rewriting since it limits a tool's scope to applications that can run on such engines.

This is where whamm! comes in. This DSL abstracts above the instrumentation technique to enable developer tooling to support a broad domain of applications while leveraging runtime capabilities as-available without reimplementation. With whamm! you can write instrumentation once and support wide domain of apps. Use engine instrumentation capabilities as available. Use bytecode rewriting to support everything else.

Bytecode Rewriting

rewriting.png

When this strategy is used, the target application bytecode must also be provided to the Whamm! compiler. To perform the bytecode rewriting injection strategy, whamm! leverages the orca Rust library. This library loads the app.wasm module into an AST representation that can then be traversed and manipulated to directly inject the instrumentation into the application's bytecode. Read more about the low-level details in the developers documentation.

The now-instrumented application is output as a new Wasm module, as shown above (app.out.wasm).

Direct Engine Support

engine.png

Users may also target an engine that directly supports instrumentation via the interface used by Whamm!. Currently, the only Wasm engine that does this is Wizard. You can read about Wizard's instrumentation capabilities in the ASPLOS paper: Flexible Non-intrusive Dynamic Instrumentation for WebAssembly.

The Whamm! compiler generates a Wasm module that encodes instrumentation in a generic way, it is not directly tied to an application's bytecode. Read the developer's Engine Target documentation for more information on how this works if interested.

Note, when using this strategy, the target application's bytecode is not provided to the Whamm! compiler. This is because the engine finds the appropriate instrumentation injection points in the application and attaches the appropriate callbacks at application runtime. Thus, the app.wasm is provided as a "co-module" (meaning a Wasm module that is directly connected to its supporting instr.wasm) to the engine at runtime.

Examples

Here are some documented examples of probes written in the language.

Branch Monitor

Here is an example monitor that can be written in Whamm!, it does not require an instrumentation library. Rather, it uses the DSL to express all of its monitoring logic.

wasm::br:before {
    report unshared i32 taken;
    // branch always taken for `br`
    // count stores an array of counters
    taken++;
}

wasm::br_if:before {
    report unshared i32 taken;
    report unshared i32 not_taken;
    
    // which branch was taken?
    if (arg0 != 0) { // arg0: Wasm top-of-stack
        taken++;
    } else {
        not_taken++;
    }
}

wasm::br_table:before {
    report unshared map<i32, i32> taken_branches;
    
    // which branch was taken?
    i32 index;
    
    // arg0: the Wasm top-of-stack
    // num_targets: the number of targets for this br_table
    // targets: the branches that can be targeted by this br_table
    // default_target: the default target for this br_table
    index = arg0 < (num_targets - 1) ? targets[arg0] : default_target;
    
    taken_branches[index]++;
}

For whamm! Developers

Do you want to contribute to whamm! or just learn about the low-level details for fun? Then you're in the right place.

Resources

Parsing:

whamm! Implementation Concepts

The four phases of compilation:

  1. Parse
  2. Configure the Whamm! Core Library (if needed)
  3. Verify
  4. Translate AST into the injection strategy's representation
  5. Emit

Other helpful concepts:

The Four Phases of Compilation

First, what is meant by the term "compilation" depends on the selected injection strategy.

For bytecode rewriting, compilation means generating a new instrumented variation of the application's bytecode.

For direct engine support, compilation means compiling the .mm script to a new Wasm module that interfaces with an engine to instrument the program dynamically. The original program is not touched and not provided when using this strategy.

The first three phases of whamm! compilation are identical for both strategies. The translate and emit phases vary between injection strategy. This is because "emitting" for bytecode rewriting means using the orcs library to insert new instructions into the program. Whereas "emitting" for direct engine support means emitting a Wasm module encoding where to instrument and the callbacks to attach at the probed sites by interfacing with the engine at application runtime.

These are the four phases of compilation:

  1. Parse
  2. Configure the Whamm! Core Library (if needed)
  3. Verify
  4. Translate AST into the injection strategy's representation
  5. Emit

Phase 1: Parse

Here is documentation describing how we parse .mm scripts.

The Grammar

whamm.pest

whamm!'s grammar is written using the Pest parser generator Rust library, which uses Parsing Expression Grammars (PEG) as input. Reading the Pest book first will inform how to read the whamm.pest grammar.

Pest parses a passed .mm script and creates a set of matched Rules that are then traversed in the whamm_parser.rs to generate whamm!'s Abstract Syntax Tree (AST). These Rules correspond to the naming used in the whamm.pest grammar.

The logic for creating the AST from the Pest Rules can be followed by starting at the parsing entrypoint: the parse_script function found in the whamm_parser.rs file.

The Abstract Syntax Tree (AST)

We use an AST to represent the .mm script after parsing. This AST is leveraged in different ways for each of the subsequent compiler phases.

During verification, the AST is used to build the SymbolTable and perform type checking.

While translating the AST into the injection strategy's representation, the AST is visited and restructured in a way that is simpler to compile for each strategy. Each node contains new data unique to each strategy that is helpful while emitting.

While emitting, the simpler AST variation mentioned above is used to lookup global statements and iterate over probe definitions to inject them into locations-of-interest in the Wasm program.

Phase 2: Configure The Whamm! Core Library

TODO

Phase 3: Verify

Here is documentation describing how we verify .mm scripts.

The SymbolTable

During verification, first the SymbolTable is build from the AST. There are great resources online that teach about symbol tables if that is helpful for any readers.

At a high-level, the SymbolTable stores metadata about source code symbols. For .mm scripts, symbols can be parts of the probe match rule (e.g. provider, package, event, and mode), function names, and variables (local or global). The enum named Record in the verifier/types.rs file defines these symbols and the metadata-of-interest for each of them.

The metadata-of-interest tends to be type information and addressing that corresponds to the ID assigned to the symbol after it's been emitted into the Wasm program.

Type information is used when type checking the script and when emitting the instrumentation (to know the type of each item being emitted).

An example of when the addressing metadata is used is emitting and calling functions. After a function defined by the instrumentation has been injected, the ID for this function would need to be stored in the SymbolTable to be looked up and used when there a call to the function is being emitted at some future point (see the InitGenerator documentation).

These symbols are contained within some scope. The types of scopes present in a .mm script can be found in the enum named ScopeType in the verifier/types.rs file. The concept of scopes in this context is the same as in other programming languages. A scope defines where variables and functions are accessible based on their location in a program.

In the context of whamm! there are some scopes that exist but aren't accessible to the end-user. Consider the probe match rule: provider:package:event:mode. Each part of this match rule really has its own scope. This enables each part to introduce its own helpful global variables and functions that the user can leverage to write more expressive instrumentation! These provided globals and functions are added to the AST in the whamm_parser.rs file. See the probes syntax documentation for a helpful CLI tool that enables the user to see what is in-scope for any given probe match rule.

Problems / Workarounds

1. Ownership of Records and Scopes.

When writing the SymbolTable structure, there were issues with pushing the ownership of Records and Scopes down into each parent. The workaround was to hold a Vec of all Records and Scopes for the entire program in the SymbolTable struct, then hold usize types that indexes into these Vecs in the Records and Scopes.

It is possible that this could be avoided by just boxing the values in the Records and Scopes, but some experimentation needs to be done.

Building the SymbolTable

The builder_visitor.rs file builds the SymbolTable from the script's AST. The visit_whamm function is the entrypoint for this behavior. This follows the visitor software design pattern. There are great resources online that teach about the visitor pattern if that is helpful for any readers.

The TypeChecker

The type checker then visits the AST and uses the SymbolTable to verify that variable usage is appropriate. It can find out-of-scope usages, invalid method invocations, misused types, etc.

Phase 4: AST Translation

TODO

Phase 5: Emit

Here is documentation describing how we emit .mm scripts.

Some Helpful Concepts

What is a generator? A generator is used to traverse some representation of logic in an abstract way. It then calls the emitter when appropriate to actually emit the code in the target representation.

What is an emitter? The emitter exposes an API that can be called to emit code in the target representation. There will be as many emitters as there are target representations supported by the language.

The Injection Strategies

The code that is emitted, and the methodology in which emitting happens, depends on the injection strategy specified by the user.

There are currently two supported injection strategies:

  1. Bytecode Rewriting
  2. Interfacing with an engine

Bytecode Rewriting

For bytecode rewriting, there are two generators. Each of these generators are used for a specific reason while emitting instrumentation. The InitGenerator is run first to emit the parts of the .mm script that need to exist before any probe actions are emitted, such as functions and global state. The InstrGenerator is run second to emit the probes while visiting the app.wasm bytecode (represented as an in-memory IR).

Both of these generators use the emitter that emits Wasm code. The emitter uses utilities that centralize the Wasm emitting logic found at utils.rs

1. InitGenerator

The init_generator.rs traverses the AST to emit functions and globals that need to exist before emitting probes. The run function is the entrypoint for this generator. This follows the visitor software design pattern. There are great resources online that teach about the visitor pattern if that is helpful for any readers.

This generator emits new Wasm functions and globals into the program with associated Wasm IDs. These IDs are stored in the SymbolTable for use while running the InstrGenerator. When emitting an instruction that either calls an emitted function or does some operation with an emitted global, the name of that symbol is looked up in the SymbolTable to then use the saved ID in the emitted instruction.

2. InstrGenerator

The instr_generator.rs calls into the emitter to gradually traverse the application in search for the locations that correspond to probe events in the .mm's AST. When a probed location is found, the generator emits Wasm code into the application at that point through emitter utilities.

Constant Propagation and Folding!!

Constant propagation and folding are a compiler optimizations that serve a special purpose in whamm!. There are lots of resources online explaining these concepts if that would be useful to the reader.

The whamm info command helps users see various globals that are in scope when using various probe match rules. All of these global variables are defined by whamm!'s compiler and should only be emitted as constant literals. If the variable were ever directly emitted into an instrumented program, with no compiler-provided definition, the program would fail to execute since the variable would not be defined.

whamm! uses constant propagation and folding to remedy this situation!

The define function in visiting_emitter.rs is how compiler constants are defined while traversing the application bytecode. These specific globals are defined in the emitter since their definitions are tied to locations in the Wasm program being instrumented.

The ExprFolder in folding.rs performs constant propagation and folding on expressions.

When considering a predicated probe, this behavior can be quite interesting. Take the following probe definition for example:

wasm:bytecode:call:alt /
    target_fn_type == "import" &&
    target_imp_module == "ic0" &&
    target_fn_name == "call_perform"
/ { ... }

All three of the globals used in the predicate are statically defined by the compiler and are provided by the call event. This means that all of these variable uses will be replaced by constants and the predicate will fold to a true or false. If the predicate folds to true, the probe actions can be emitted at the found location without condition. If the predicate folds to false, the probe should not be emitted.

Now, take the next probe definition example:

wasm:bytecode:call:alt /
    target_fn_type == "import" &&
    target_imp_module == "ic0" &&
    target_fn_name == "call_new" &&
    strcmp((arg0, arg1), "bookings") &&
    strcmp((arg2, arg3), "record")
/ { ... }

The predicate of this probe now includes both variables that are defined statically and variables that are defined dynamically, which is totally valid semantically!

So, what happens here? The first three globals will be propagated away to constants, the expression will be folded, and those constant equivalence checks will evaluate to either true or false. This reduced value will then be and-ed together with the following dynamically-defined portion of the expression. So, the same goal will be accomplished here as in the previous example (the probe either will or will not be emitted at that bytecode location based on statically determined information). However, this time the actions emitted will retain a conditional, but it will be the folded conditional that only includes the dynamic portion of the original predicate.

Pretty cool, right??

Interfacing with an Engine

TODO

The whamm! CLI

The CLI is defined using the clap Rust crate.

The .wast Test Harness

A .wast file is used for testing purposes and simplifies the writing of tests for developers. We use .wast files to encode assertions that should pass when running an instrumented variation of a wasm module.

Writing .wast Tests

The high level structure looks like this:

<module_in_wat>

;; WHAMM --> <some_oneline_whamm_script>
<whamm0_assertion0> ;; The first assertion for the first whamm script
<whamm0_assertion1> ;; The second assertion for the first whamm script

;; WHAMM --> <some_oneline_whamm_script>
<whamm1_assertion0> ;; The first assertion for the second whamm script
<whamm1_assertion1> ;; The second assertion for the second whamm script
<whamm2_assertion1> ;; The third assertion for the second whamm script

The module encoded in the script above would be used for all the following whamm/assertion groups. To verify that all assertions fail before instrumenting, 5 new .wast files would be generated with the original module and run on the configured interpreters. To verify that all assertions pass after instrumenting, 2 new .wast files would be generated, one per specified whamm script, including the assertions under the respective whamm script.

Below is an example .wast test:

;; Test `wasm:opcode:call` event

;; @instrument
(module
    ;; Auxiliary definitions
    (func $other (param i32) (result i32) (local.get 1))
    (func $dummy (param i32) (result i32) (local.get 0))

    ;; Test case functions
    (func (export "instrument_me") (result i32)
        (call $dummy (i32.const 0))
    )
)

;; WHAMM --> wasm:opcode:call:before { arg0 = 1; }
(assert_return (invoke "instrument_me") (i32.const 1)) ;; will be run with the above WHAMM instrumentation

;; WHAMM --> wasm:opcode:call:alt { alt_call_by_name("other"); }
(assert_return (invoke "instrument_me") (i32.const 1)) ;; will be run with the above WHAMM instrumentation

Below is an example .wast test using imports:

(module
    (func (export "dummy") (param i32) (result i32)
        local.get 0
    )
)

(register "test")

;; @instrument
(module
    ;; Imports
    (type (;0;) (func (param i32) (result i32)))
    (import "test" "dummy" (func $dummy (type 0)))

    ;; Globals
    (global $var (mut i32) (i32.const 0))

    ;; Global getters
    (func $get_global_var (result i32)
        (global.get $var)
    )

    ;; Test case functions
    (func $foo
        (call $dummy (i32.const 0))
        global.set $var
    )

    (start $foo)
    (export "foo" (func $foo))
    (export "get_global_var" (func $get_global_var))
    (memory (;0;) 1)
 )
 
;; WHAMM --> i32 count; wasm:opcode:call:alt / arg0 == 0 / { count = 5; return 1; }
(assert_return (invoke "get_global_var") (i32.const 1)) ;; alt, so global should be return value
(assert_return (invoke "get_count") (i32.const 5))

There are several conventions to follow when writing .wast test cases for whamm.

  1. Only one module-to-instrument per .wast file.
    • The test setup goes at the top (which can include multiple modules when considering testing imports).
    • The module-to-instrument is the final part of the setup and is marked by ;; @instrument above the module.
  2. Use comment to specify the whamm! script, syntax: ;; WHAMM --> <whamm_script>
    • The scripts are run on the module in the .wast file.
    • If there are multiple asserts under a whamm! comment, they are all run against the instrumented variation of the module that results from that whamm! script.
  3. All asserts should fail if they were to run without instrumentation.

NOTE: For wizard, don't do manipulations that change arg* (that requires the frame accessor). Instead change global state for now?

The Harness Code

The harness is located in tests/common/wast_harness.rs with the main function as the entrypoint. We invoke this harness through calling the main entrypoint in the run_wast_tests test case located in tests/integration_test.rs.

One can read the harness code and see that it performs the following logic:

  1. Split out test components of each .wast file found under tests/wast_suite as an individual WastTestCase
    • a wasm module
    • a whamm script
    • a list of assertions that should be true post-instrumentation
  2. Ensure all assertions fail before instrumenting with whamm. We do this to be able to claim that correctness of instrumentation was the sole purpose that some test passed. We ensure that this property holds by first creating new .wast files with one assertion per file and making sure that they fail when run on a supported interpreter.
    • We do this re-generation of the .wast files because the interpreters we use exit on the first failed assertion per .wast, but we want to guarantee this property for all assertions.
  3. Ensure all assertions pass after instrumenting with whamm.
    • Run the specified whamm script on the module per set of assertions.
    • Output a new .wast file with the instrumented variation of the module with the respective assertions.

Supported Interpreters

The harness generates *.bin.wast files to run on a list of engines, e.g. wizeng and the spec interpreter.

See the repo's README.md for how to set up the interpreters to run with our test harness.

Some Ideas for Future Improvements

Report Variables

;; Use something like below to assert on the values of some report variable dynamically.
;; REPORT_TRACE(ID) --> 1, 3, 5, 6, 7

;; Use something like below to assert on report variable values!
;; WITH_WHAMM --> (assert_return (invoke "get_report_var" (i32.const 1)) (i32.const 7))

Error Handling

Errors are added to ErrorGen, defined in error.rs, and reported between compiler phases. If an error occurs during a compilation phase, the errors are reported and the compilation is aborted.

Contributors

These are the people to thank when you're using whamm!...either genuinely or sarcastically...if you have problems, you should let them know in a GH issue ;)

Elizabeth Gilbert, PhD student at Carnegie Mellon University (CMU).

Alex Bai, undergrad student at Tufts University.

Wavid Bowman, undergrad student at University of Florida.