Istio ALPN#
Protocol sniffing#
Background#
Istio 1.3 releases the protocol sniffing for outbound traffic. There are two remaining work:
Enable protocol sniffing for inbound listener.
Enable protocol sniffing for HTTP 2 traffic
The document includes some decisions that used for better networking.
Current inbound listener filter working flow, considering both TLS inspector and HTTP inspector are used.
Client Sidecar used |
Application Protocols |
Traffic over mTLS |
TLS Inspector Output |
HTTP Inspector Output |
Output |
---|---|---|---|---|---|
Yes |
TCP |
No |
N/A |
ALPN: [] |
ALPN: [] |
Yes |
TCP |
Yes |
ALPN: [“istio”]Transport Protocol: tls |
N/A |
ALPN: [“istio”]Transport Protocol: tls |
Yes |
HTTP 1.x |
No |
N/A |
ALPN: [“HTTP 1.x”] |
ALPN: [“HTTP 1.x”] |
Yes |
HTTP 1.x |
Yes |
ALPN: [“istio”]Transport Protocol: tls |
N/A |
ALPN: [“istio”]Transport Protocol: tls |
No |
TCP |
No |
N/A |
ALPN: [] |
ALPN: [] |
No |
TCP |
Yes |
ALPN: []Transport Protocol: tls |
N/A |
ALPN: []Transport Protocol: tls |
No |
HTTP 1.x |
No |
N/A |
ALPN: [“HTTP 1.x”] |
ALPN: [“HTTP 1.x”] |
No |
HTTP 1.x |
Yes |
ALPN: []Transport Protocol: tls |
N/A |
ALPN: []Transport Protocol: tls |
When traffic is over TLS and TLS inspector is used by the listener, HTTP inspector will be skipped. HTTP inspector cannot be used to detect ALPN (HTTP vs TCP) from the incoming packet and cannot select filter chain. If client doesn’t provide enough ALPN for server to select filter chain, the TLS inspector cannot select the correct filter chain.
Objective#
The doc is going to
Describe several alternatives for resolving protocol sniffing when traffic is over TLS and TLS inspector is used.
Describe the solution for using protocol sniffing for h2.
Overview#
Protocol sniffing for inbound listener#
Filter chains#
In order to support sniffing on inbound, we need 4 inbound filter chains.
Transport Protocol |
Application Protocol |
Remarks |
|
---|---|---|---|
FCM1 |
tls |
[“istio-http/1.1”, “istio-h2”] |
“istio-http/1.1” is for client uses sidecar + http/1.1“Istio-h2” is for client uses sidecar + h2 |
FCM2 |
empty |
[“http/1.1”, “h2c”] |
|
FCM3 |
tls |
[“istio”] |
This is for traffic over TLS from client with sidecar |
FCM4 |
empty |
[] |
|
FCM5 |
tls |
[] |
This branch includes the case that the client doesn’t use sidecar and sends HTTP traffic over TLS. |
Get ALPN#
Make Envoy client inject the ALPN into the TLS context according to the different upstream connections. E.g., for http/1.1 connection over TLS, the Envoy will inject istio-http/1.1 in the TLS context and the server will inspect the istio-http/1.1 and select the http filter chain over TLS. If the client doesn’t have Envoy sidecar, the inspector relies on whether the client set ALPN or not. If the client doesn’t set ALPN, the traffic will be downgraded to TCP.
The ALPN filter will provide the context like:
alpn_override:
- upstream_protocol: HTTP10
alpn_override: ["istio-http/1.0", "istio"]
- upstream_protocol: HTTP11
alpn_override: ["istio-http/1.1", "istio"]
- upstream_protocol: HTTP2
alpn_override: ["istio-h2", "istio"]
The reason for including “istio” in the alpn_override is for backward compatible.
Alternatives Considered#
Inspect ALPN using HTTP inspector when TLS is enabled before creating the filter chain. An alternative to this solution would be creating two listeners, the first listener for terminating TLS connection and use the second listener to do ALPN sniffing.
Solution |
Pros |
Cons |
---|---|---|
Inject ALPN in the client side |
Easy to implement in Envoy side compared with inspecting ALPN using HTTP inspector |
Need extra work to handle client without sidecar. (The server could use TCP proxy for traffic from client without sidecar). Perhaps we can rely on client to set ALPN itself. |
Inspect ALPN after TLS inspector |
Handle traffic from all clients |
With current framework, it seems not possible to inspect ALPN with http inspector over TLS. Need lots of effort to work on Envoy |
Use two listeners |
No extra effort on Envoy |
Performance degradation |