Try Edge Computing With Envoy
A Demo of solving business problems with Envoy and WASM
Welcome! This guide has 2 purposes:
provide a reproducible workflow to developing a WASM filter for envoy
Demonstrate how Envoy and WASM Filters can be used for edge computing applications
Warning: Envoy filter development is not for the faint of heart. This guide will give you a good intro and walk through the working example on Github. You may still encounter errors. The sources linked below may be helpful for debugging issues. Also, feel free to reach out!
This article is part of a Series on Edge Computing. For an introduction to edge computing see [the first article in the series](). For more guides like this, see the guide's home page.
The Problem
Before diving into writing an Envoy filter, let's define a real-world problem that we are going to try and solve.
Imagine we are a SaaS provider with a multi-tenant product running on Kubernetes. The problem is that the Ops team can't identify which customers are being impacted during incidents. The identified solution is to track the customer ID on each API request, allowing the Operations team to immediately know if the indcident is localized or affecting all customers. To update all the microservices to have this behavior was going to a big challenge. Instead, the business realized the Platform team could implement this solution inside their API Gateway and Service Mesh.
Since our company is running Istio, we can create an Envoy filter (more about this in the next section). This filter will look up the customer ID using the authorization header and update the response headers. Then envoy will add this custom header to the OpenTelemetry traces, allowing the Ops team to access the data they need.
Design
Before we implement the solution let's understand the key parts of envoy.
Envoy processes each request through a Listener. A listener defines a "chain" of filters, where each filter processes the request before the request is passed to the next filter. Most filters are written and compiled into Envoy. But one filter, the WASM filter, allows developers to write a custom envoy filter without having to compile Envoy.
The WASM filter accepts a WASM module and executes it for every request it processes. WASM modules are written with the proxy-wasm SDK. Let's get to writing our filter now.
Implement the Filter
We're going to develop an envoy filter in Rust. Rust has seen a lot of attention in the WebAssembly ecosystem. The Envoy project is no exception and we can expect fairly good support for rust. Consider cloning the Github repo, then using this guide as an in-depth explanation of the code.
Local Dev Setup
First, let's install Rust and Cargo, the Rust package manager/task runner. We'll use the language instructions here. The only additional step is that we need to install the WASM32 compilation target using the following command:
rustup update
rustup target add wasm32-unknown-unknown
Create The Project
Now we'll create a normal rust project using the Cargo CLI:
$ cargo new --lib envoy-filter-customer-lookup
Created library `envoy-filter-customer-lookup` package
Next, we make 2 updates to Cargo.toml. Add the following dependencies:
log = "0.4.8"
proxy-wasm = "0.2.1"
We also need to tell Rust that this project will be compiled as a dynamic library:
[lib]
path = "src/lib.rs"
crate-type = ["cdylib"]
The full Cargo.toml file is on Github.
Code the Filter
Here is a "Hello World" envoy filter:
use log::info;
use proxy_wasm as wasm;
#[no_mangle]
pub fn _start() {
proxy_wasm::set_log_level(wasm::types::LogLevel::Trace);
proxy_wasm::set_http_context(
|context_id, _root_context_id| -> Box<dyn wasm::traits::HttpContext> {
Box::new(HelloWorld { context_id })
},
)
}
struct HelloWorld {
context_id: u32,
}
impl wasm::traits::Context for HelloWorld {}
impl wasm::traits::HttpContext for HelloWorld {
fn on_http_request_headers(&mut self, num_headers: usize, _end_of_stream: bool) -> wasm::types::Action {
info!("Got {} HTTP headers in #{}.", num_headers, self.context_id);
let headers = self.get_http_request_headers();
let mut authority = "";
for (name, value) in &headers {
if name == ":authority" {
authority = value;
}
}
self.set_http_request_header("x-hello", Some(&format!("Hello world from {}",
authority)));
wasm::types::Action::Continue
}
}
Here's a simple breakdown of the code:
First, we create a "HelloWorld" struct to implement 2 traits against
We will implement one method on
wasm::traits::Contextlater, that's why we declared itThe
on_http_request_headersmethod gets called when the envoy filter has received the HTTP headersIn this method, we grab the
:authorityheader and use it to create a new header on the requestFinally, our method returns
wasm::types::Action::Continueto tell Envoy to keep processing the filter
Next, let's compile.
$ cargo build --target wasm32-unknown-unknown --release
Note: I got a error that looked like = note: xcrun: error: invalid active developer path (. This just meant that I needed to update XCode. Do that with xcode-select --install.
Deploy our filter inside Envoy
Let's test out the filter. Envoy provides docker based "Sandboxes," where there is a pre-configured envoy service that we can modify to suit our needs. I started with this sandbox. You can view the full configuration in Github here.
We'll update the docker-compose.yml to look like this:
services:
proxy:
image: envoyproxy/envoy:dev
volumes:
- ${PWD}/docker/envoy.yaml:/etc/envoy.yaml
- ${PWD}/target/wasm32-unknown-unknown/release/envoy_filter_customer_lookup.wasm:/lib/envoy_filter_customer_lookup.wasm
depends_on:
- web_service
ports:
- "8000:8000"
command: /usr/local/bin/envoy -c /etc/envoy.yaml --service-cluster proxy -l info
web_service:
image: ealen/echo-server:0.7.0
ports:
- 80:80cp target/wasm32-unknown-unknown/release/envoy_filter_customer_lookup.wasm ./
This gets the Envoy container to mount our compiled WASM module.
The HTTP Filter configuration for Envoy looks like this:
http_filters:
- name: envoy.filters.http.wasm
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
config:
name: "my_plugin"
root_id: "my_root_id"
# if your wasm filter requires custom configuration you can add
# as follows
configuration:
"@type": "type.googleapis.com/google.protobuf.StringValue"
value: |
{}
vm_config:
vm_id: "my_vm_id"
code:
local:
filename: "lib/envoy_filter_customer_lookup.wasm"
Don't forget to copy the compiled envoy filter to the local directory so that we docker can mount it:
cp target/wasm32-unknown-unknown/release/envoy_filter_customer_lookup.wasm ./
Bring up the docker containers and send a request:
docker compose up -d
curl localhost:8000
If we view the Envoy container's logs, we can see our WASM filter at work.
$ docker compose logs --follow
...
proxy_1 | [2023-05-30 20:48:15.240][21][info][wasm] [source/extensions/common/wasm/context.cc:1148] wasm log my_plugin my_root_id my_vm_id: Got 8 HTTP headers in #2.
proxy_1 | [2023-05-30T20:48:15.239Z] "GET / HTTP/1.1" 200 - 0 627 36 35 "-" "curl/7.65.1" "ad277cc6-fb8b-4966-abc9-c6e8a739a069" "localhost:8000" "192.168.0.2:80"
Now we have a local development cycle to build our filter. Next, let's implement the solution that will send an HTTP request to the customer API and decorate the response headers with the customer ID.
Implement the Filter
For the sake of this guide, I built a simple Customer API. It's wired up in the docker-compose.yml and you can see the implementation here. In this section we'll wire up the customer API to our Envoy instance and then code the filter.
To allow our Envoy filter to communicate to it, we have to add an Envoy "cluster" to our config.
- name: customer_api
connect_timeout: 5s
type: strict_dns
lb_policy: round_robin
load_assignment:
cluster_name: customer_api
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
protocol: TCP
address: customer_api
port_value: 3030
Now, let's code the filter. First, we must extract the auth header from the request headers, then issue an HTTP request to the customer API. We'll do this in the on_http_request_headers() method:
fn on_http_request_headers(&mut self, num_headers: usize, _end_of_stream: bool) -> wasm::types::Action {
info!("Got {} HTTP headers in #{}.", num_headers, self.context_id);
let headers = self.get_http_request_headers();
let mut token = "";
// grab the auth header
for (name, value) in &headers {
if name == ":Authorization" {
token = value;
}
}
// issue HTTP request to customer API
let res = self.dispatch_http_call(
"customer_api",
vec![
(":method", "GET"),
(":path", "/api/lookup"),
(":authority", "customer_api"),
("Authorization", token)],
None,
vec![], Duration::from_millis(500)).unwrap();
// stop further processing of this request
wasm::types::Action::Pause
}
Let's explain this code further. In Envoy, HTTP requests are issued via the dispatch_http_call() method. The method doesn't have a typical signature - this is one of the idiosyncrasies of developing in WebAssembly environments.
Finally, the on_http_request_headers() method returns wasm::types::Action::Pause, telling Envoy to not pass this request onto the next filter.
Next, we need to write some code to handle the response of the HTTP request we just issued. We will handle this in the on_http_call_response() method of the wasm::traits::Context trait. This method gets called whenever Envoy receives the response. Remember: our WASM code didn't origin the HTTP request, it made a request to the WASM host - the WASM filter - to send an HTTP request. The WASM host will then give our code the response, when it's available. Here's the code:
impl wasm::traits::Context for HelloWorld {
fn on_http_call_response(&mut self, _token_id: u32,
_num_headers: usize, body_size: usize, _num_trailers: usize,) {
info!("http call completed");
match self.get_http_call_response_body(0, body_size) {
Some(body) => {
match std::str::from_utf8(&body) {
Ok(v) => {
info!("Body: {}", v);
let value = self.get_customer_id(v);
info!("customer id: {}", value);
self.customer_id = value;
}
Err(e) => info!("Invalid UTF-8 sequence: {}", e),
};
},
None => info!("failed to get response body")
}
self.resume_http_request();
}
}
There's a lot of nested pattern matching going on here, and that's simply to account for the possible failures scenarios. There may be a better way to code this in Rust. Here's a breakdown of the code:
first, we request the HTTP body
Next, we try to convert the byte steam to a UTF-8 string
If that's successful, we call
self.get_customer_id()which simply extracts the ID from the JSON response of the Customer API.We'll store this ID using
self.customer_id = value, this ensures that we can access this customer ID at a later stepVery important: at the end, we call
self.resume_http_request()to ensure that envoy proceeds processing the request
Here's how we implemented the self.get_customer_id() method:
#[derive(Deserialize, Debug)]
struct CustomerLookupResponse {
id: u32,
name: String
}
impl HelloWorld {
fn get_customer_id(&mut self, raw: &str) -> String {
let r: serde_json::error::Result<CustomerLookupResponse> = serde_json::from_str(raw);
info!("parsed json: {:#?}", r);
r.map_or(String::from("unknown"), |res| res.id.to_string())
}
}
This code simply attempts to marshal the raw HTTP response into the CustomerLookupResponse type, and if successful, return the id. Otherwise, it will return unknown. The last change necessary is to update put the customer ID in the response headers of the request.
To modify response headers, we'll use another method on wasm::traits::HTTPContext: on_http_response_headers().
fn on_http_response_headers(&mut self, _num_headers: usize, _end_of_stream: bool) -> wasm::types::Action {
self.set_http_response_header("x-customer-id", Some(&self.customer_id.as_str()));
wasm::types::Action::Continue
}
In this method, we're simply adding the header, and again, returning the Continue action. We've done it! Now let's try it out.
Test the Filter
If you haven't already done so, start the Envoy sandbox with docker compose up -d.
Compile our code into a WASM module with cargo build --target wasm32-unknown-unknown --release.
Now we can finally issue an HTTP request against our sandbox and see the x-customer-id header being added:
a-cusick@NFIT-cusick envoy-filter-customer-lookup % curl -i -H "Authorization: Bearer asdfzxcv" localhost:8000
HTTP/1.1 200 OK
content-type: application/json; charset=utf-8
content-length: 619
etag: W/"26b-5e7bcwjiYoPrHjlpoapllr3LWCA"
date: Sat, 10 Jun 2023 08:18:27 GMT
x-envoy-upstream-service-time: 4
x-customer-id: 1
server: envoy
Violá!
The envoy logs are very helpful in understanding the behavior of your WASM filter:
In these logs, we can see the customer API response being properly marshaled into CustomerLookupResponse.
Summary
In the beginning, we established a business case where a SaaS business's Operations team needed to see the customer ID on all requests. This would allow the Ops team to instrument their monitoring tools to show which customers are being affected during an incident. We were able to accomplish this without changing any of the application code and simply leveraging Envoy, the proxy behind Istio.
The Envoy WASM SDK allows you to code WASM modules that interface with the WASM Filter. The WASM filter acts as a WASM Host, which calls methods in our WASM module - like on_http_call_response(), on_http_request_headers(). Our WASM module also makes calls the host, like dispatch_http_call().