Deploying cache for usage with Istio:
Building a filter
Istio 1.4.2 is using a proxy based on https://github.com/istio/envoy/ commit 6d525c67f39b36cdff9d688697f266c1b55e9cb7
(sha 53efbe4a35e2c4dbee9a2ee50ce6469eb8bbcbb997bec5a2023623d60f629c0c)
To build a filter, you can start off from the example in the example/cpp-istio-1.4
folder.
build with:
bazel build :filter.wasm
push it to somewhere publicly accessible. webassemblyhub.io for example
wasme push webassemblyhub.io/yuval-k/istio-example:1.4.2 bazel-bin/filter.wasm
Build wasme cache container
Note this is only needed if you changed the cache code.
If you update the cache code, build and push docker image:
docker build -t quay.io/solo-io/wasme:0.0.1 .
docker push quay.io/solo-io/wasme:0.0.1
Deploy cache container
kubectl create namespace wasme-cache
kubectl apply -n wasme-cache -f pkg/cmd/cache/deploy.yaml
Edit deployment where filter is desired
Edit your pod to include the cached filters as a volume:
sidecar.istio.io/userVolume: '[{"name":"cache-dir","hostPath":{"path":"/var/local/lib/wasme-cache"}}]'
sidecar.istio.io/userVolumeMount: '[{"mountPath":"/var/local/lib/wasme-cache","name":"cache-dir"}]'
In istio 1.4, wasm filters only get loaded to TPROXY enabled listeners. so we'll need the following
annotations:
sidecar.istio.io/interceptionMode: TPROXY
traffic.sidecar.istio.io/includeInboundPorts: "9080"
Note: includeInboundPorts is needed as otherwise communication to pilot is sent to the TPROXY rule,
which doesn't end well for envoy. This port may need to be adjusted to your workload.
Note that these should be on the annotations of the pod template: e.g.:
apiVersion: apps/v1
kind: Deployment
metadata:
name: details-v1
labels:
app: details
version: v1
spec:
replicas: 1
selector:
matchLabels:
app: details
version: v1
template:
metadata:
annotations:
sidecar.istio.io/userVolume: '[{"name":"cache-dir","hostPath":{"path":"/var/local/lib/wasme-cache","type":"Directory"}}]'
sidecar.istio.io/userVolumeMount: '[{"mountPath":"/var/local/lib/wasme-cache","name":"cache-dir"}]',
sidecar.istio.io/interceptionMode: TPROXY
traffic.sidecar.istio.io/includeInboundPorts: "9080"
labels:
app: details
version: v1
spec:
serviceAccountName: bookinfo-details
containers:
- name: details
image: docker.io/istio/examples-bookinfo-details-v1:1.15.0
imagePullPolicy: IfNotPresent
ports:
- containerPort: 9080
you can use the following command to apply:
kubectl -n default patch deploy/details-v1 --type=merge -p='{"spec":{"template":{"metadata":{"annotations":{"traffic.sidecar.istio.io/includeInboundPorts":"9080","sidecar.istio.io/interceptionMode":"TPROXY","sidecar.istio.io/userVolume":"[{\"name\":\"cache-dir\",\"hostPath\":{\"path\":\"/var/local/lib/wasme-cache\",\"type\":\"Directory\"}}]","sidecar.istio.io/userVolumeMount":"[{\"mountPath\":\"/var/local/lib/wasme-cache\",\"name\":\"cache-dir\"}]"}}}}}'
Apply filter - EnvoyFilter CRD
Add an envoy filter CRD with your filter. By default the deployment has a test wasme filter. it is cached with this bbfdf674f5cf2e413a4b701ae865dd4569502f60af5647f2f47ca4f38e2b40af
hash.
Note: the wasme deploy
command will soon be updated to automate this step of providing the filter hash.
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: details-wasm
spec:
workloadSelector:
labels:
app: details
configPatches:
- applyTo: HTTP_FILTER
match:
context: SIDECAR_INBOUND
listener:
portNumber: 9080
filterChain:
filter:
name: "envoy.http_connection_manager"
subFilter:
name: "envoy.router"
patch:
operation: INSERT_BEFORE
value:
name: envoy.filters.http.wasm
config:
config:
name: "test"
root_id: "add_header_root_id"
configuration: '{"value":"hi-from-wasme"}'
vm_config:
runtime: envoy.wasm.runtime.v8
code:
filename:
/var/local/lib/wasme-cache/bbfdf674f5cf2e413a4b701ae865dd4569502f60af5647f2f47ca4f38e2b40af
apply with:
kubectl apply -n default -f pkg/cmd/cache/filter.yaml
Test:
kubectl exec -ti -n default deploy/productpage-v1 -c istio-proxy -- curl -v http://details.default:9080/details/123
debugging
check details config dump:
kubectl port-forward -n default deploy/details-v1 15000 &
curl http://localhost:15000/config_dump
logs from envoy:
curl 'localhost:15000/logging?level=debug' -XPOST
kubectl logs -n default deploy/details-v1 -c istio-proxy
logs from pilot (see nacks in there):
kubectl logs -n istio-system deploy/istio-pilot -c discovery