MOSN 0.1.0 performance report

Instructions on performance report

The following performance report presents the performance comparison data of MOSN 0.1.0 with envoy in terms of pure TCP forwarding for Bolt and HTTP1.x protocols, mainly including QPS, RTT, failure rate, success rate and other indicators.

It is significant to note the following optimizations in v0.1.0 which are intended to improve the forwarding performance of MOSN.

• For thread model, MOSN uses the worker goroutine pool to handle stream events, and uses two independent goroutines to handle read and write IO separately.
• For single-core forwarding, in the case of specifying P=1, MOSN binds CPU with cores to improve the call execution efficiency of the system and the locality affinity of cache.
• For memory, in the case of binding single core, MOSN uses SLAB-style recycling mechanism to improve reuse and reduce memory copy.
• For IO, MOSN mainly implements optimization by controlling the read/write buffer size, read/write timing, read/write frequency and other parameters.

The performance test data is as follows:

TCP proxy performance data

For the same deployment mode, this report compares MOSN 0.1.0 and envoy for the upper-layer protocol Bolt (SOFARPC related protocol) and HTTP1.1 respectively.

Deployment mode

The pressure test is deployed in a pure proxy mode. The client process accesses the server process through the MOSN process and serves as a forwarding proxy. The client process, MOSN process, and server process run on the machines which belong to different network segments. The network delay of the direct access from the client to server is about 2.5ms.

Client

Bolt protocol (send 1K string)

The client that sends the Bolt protocol data uses the online pressure generators developed by Ant Financial and deploys the SOFARPC client.

On the pressure generator performance page, you can see the QPS, success/failure counts, RT and other parameters.

HTTP1.1 protocol (send 1K string)

Use ApacheBench/2.3. The test instructions are:

ab -n $RPC -c $CPC -p 1k.txt -T "text/plain" -k http://11.166.161.136:12200/tcp_bench > ab.log.$CPU_IDX &

Mesh machine specifications

The mesh runs in a container where the CPU is an exclusive logical core. The specifications are as follows:

Upstream machine specifications

Bolt protocol test result

Performance data

Conclusion

For single-core TCP forwarding, there is little difference between MOSN 0.1.0 and Envoy 1.7 in terms of performance in the condition with full load, such as QPS, RTT and success/failure counts. We will continue to optimize in the subsequent versions.

HTTP/1.1 test result

Since the HTTP/1.1 request response model is PING-PONG, QPS is positively correlated with the number of concurrences. The following tests are performed for different concurrence counts.

Concurrences - 20

Concurrences - 40

Concurrences - 200

Concurrences - 220

Conclusion

When the upper-layer protocol is HTTP/1.X., a certain gap exists between MOSN and Envoy in terms of performance.

The preliminary conclusion is that in the PING-PONG package delivery model, MOSN cannot merge the read/write system calls. In the scenario where SOFARPC can merge the calls, MOSN’s HTTP performance is weaker than that of Envoy because the syscall count significantly increases.

This problem will be optimized in version 0.2.0.

Appendix

Envoy version information

• version:1.7
• Tag:1ef23d481a4701ad4a414d1ef98036bd2ed322e7

Envoy TCP test configuration

static_resources:
  listeners:
  - address:
      socket_address:
        address: 0.0.0.0
        port_value: 12200
    filter_chains:
    - filters:
      - name: envoy.tcp_proxy
        config:
          stat_prefix: ingress_tcp
          cluster: sofa_server
  clusters:
  - name: sofa_server
    connect_timeout: 0.25s
    type: static
    lb_policy: round_robin
    hosts:
    - socket_address:
        address: 10.210.168.5
        port_value: 12222
    - socket_address:
        address: 10.210.168.5
        port_value: 12223
    - socket_address:
        address: 10.210.168.5
        port_value: 12224
    - socket_address:
        address: 10.210.168.5
        port_value: 12225
admin:
  access_log_path: "/dev/null"
  address:
    socket_address:
      address: 0.0.0.0
      port_value: 8001