### NEW !!! 👉 [SRPC tools : build Workflow and SRPC projects easily.](/tools/README.md)
## Introduction
**SRPC is an enterprise-level RPC system used by almost all online services in Sogou. It handles tens of billions of requests every day, covering searches, recommendations, advertising system, and other types of services.**
Bases on [Sogou C++ Workflow](https://github.com/sogou/workflow), it is an excellent choice for high-performance, low-latency, lightweight RPC systems. Contains AOP aspect-oriented modules that can report Metrics and Trace to a variety of cloud-native systems, such as OpenTelemetry, etc.
Its main features include:
* Support multiple RPC protocols: [`SRPC`](/tutorial/tutorial-01-srpc_pb_server.cc), [`bRPC`](/tutorial/tutorial-05-brpc_pb_server.cc), [`Thrift`](/tutorial/tutorial-07-thrift_thrift_server.cc), [`tRPC`](/tutorial/tutorial-11-trpc_pb_server.cc)
* Support multiple operating systems: `Linux`, `MacOS`, `Windows`
* Support several IDL formats: [`Protobuf`](/tutorial/echo_pb.proto), [`Thrift`](/tutorial/echo_thrift.thrift)
* Support several data formats transparently: `Json`, `Protobuf`, `Thrift Binary`
* Support several compression formats, the framework automatically decompresses: `gzip`, `zlib`, `snappy`, `lz4`
* Support several communication protocols transparently: `tcp`, `udp`, `sctp`, `tcp ssl`
* With [HTTP+JSON](/docs/docs-07-srpc-http.md), you can communicate with the client or server in any language
* Use it together with [Workflow Series and Parallel](/docs/docs-06-workflow.md) to facilitate the use of calculations and other asynchronous resources
* Perfectly compatible with all Workflow functions, such as name service, [upstream](docs/docs-06-workflow.md#3-upstream) and other components
* Report [Tracing](/docs/docs-08-tracing.md) to [OpenTelemetry](https://opentelemetry.io)
* Report [Metrics](/docs/docs-09-metrics.md) to [OpenTelemetry](https://opentelemetry.io) and [Prometheus](https://prometheus.io)
* [More features...](/docs/en/rpc.md)
## Installation
srpc has been packaged for Debian and Fedora. Therefore, we can install it from source code or from the package in the system.
reference: [Linux, MacOS, Windows Installation and Compilation Guide](docs/en/installation.md)
## Quick Start
Let's quickly learn how to use it in a few steps.
For more detailed usage, please refer to [Documents](/docs) and [Tutorial](/tutorial).
#### 1\. example.proto
~~~proto
syntax = "proto3";// You can use either proto2 or proto3. Both are supported by srpc
message EchoRequest {
string message = 1;
string name = 2;
};
message EchoResponse {
string message = 1;
};
service Example {
rpc Echo(EchoRequest) returns (EchoResponse);
};
~~~
#### 2\. generate code
~~~sh
protoc example.proto --cpp_out=./ --proto_path=./
srpc_generator protobuf ./example.proto ./
~~~
#### 3\. server.cc
~~~cpp
#include
### NEW !!! 👉 [SRPC tools : 一个帮你快速构建Workflow和SRPC项目的小工具.](/tools/README_cn.md)
## Introduction
**SRPC是全搜狗业务线上使用的企业级RPC系统，目前每天承载上百亿的请求量，涵盖搜广推及其他类型业务。**
底层基于[Sogou C++ Workflow](https://github.com/sogou/workflow)，是高性能、低延迟、轻量级RPC系统的极佳选择。且加入了AOP面向切面的模块，支持Metrics（监控指标）和Trace（链路追踪）功能，可上报到多种云原生系统，包括OpenTelemetry。
主要功能和示例：
* 支持多种RPC协议：[`SRPC`](/tutorial/tutorial-01-srpc_pb_server.cc)、[`bRPC`](/tutorial/tutorial-05-brpc_pb_server.cc)、[`Thrift`](/tutorial/tutorial-07-thrift_thrift_server.cc)、[`tRPC`](/tutorial/tutorial-11-trpc_pb_server.cc)
* 支持多种操作系统：`Linux` / `MacOS` / `Windows`
* 支持多种IDL格式：[`Protobuf`](/tutorial/echo_pb.proto)、[`Thrift`](/tutorial/echo_thrift.thrift)
* 支持多种数据布局，使用上完全透明：`Json`、`Protobuf`、`Thrift Binary`
* 支持多种压缩，框架自动解压：`gzip`、`zlib`、`snappy`、`lz4`
* 支持多种通信协议：`tcp`, `udp`, `sctp`, `tcp ssl`
* 可以通过[http+json实现跨语言](/docs/docs-07-srpc-http.md)
* 可以使用[Workflow串并联任务流](/docs/docs-06-workflow.md)，打通计算及其他异步资源
* 完美兼容Workflow所有功能：命名服务体系、[upstream](docs/docs-06-workflow.md#3-upstream)、其他组件等
* 链路追踪功能：上报[Tracing](/docs/docs-08-tracing.md)到[OpenTelemetry](https://opentelemetry.io)
* 监控功能：上报[Metrics](/docs/docs-09-metrics.md)到OpenTelemetry和[Prometheus](https://prometheus.io)
## Installation
SRPC是Debian Linux和Fedora的自带安装包，因此可以通过源码安装，和系统自带安装包安装。
具体参考：[Linux、MacOS、Windows安装和编译指南](docs/installation.md)
## Quick Start
我们通过几个步骤快速了解如何使用。
更详细的用法可以参考[更多文档](/docs)，和[官方教程](/tutorial)。
#### 1. example.proto
~~~proto
syntax = "proto3";//这里proto2和proto3都可以，srpc都支持
message EchoRequest {
string message = 1;
string name = 2;
};
message EchoResponse {
string message = 1;
};
service Example {
rpc Echo(EchoRequest) returns (EchoResponse);
};
~~~
#### 2. generate code
~~~sh
protoc example.proto --cpp_out=./ --proto_path=./
srpc_generator protobuf ./example.proto ./
~~~
#### 3. server.cc
~~~cpp
#include 
我们可以看到上图client这边的span_id: **04d070f537f17d00**，它在下图server这里变成了parent_span_id: **04d070f537f17d00**：
### 2. Traces上报到Jaeger
打开Jaeger的主页，我们可以找到我们名为**Example**的服务(service)和名为**Echo**的函数(method)。这一个tracing记录上有两个span节点，是由server和client分别上报的。
我们可以在Jaeger看到client所上报的span_id: **04d070f537f17d00**和server所上报的span_id: **00202cf737f17d00**，同时也是能对应上刚才在屏幕看到的id的。
### 3. 参数
多久收集一份trace信息、上报请求的重试次数、以及其他参数，都可以通过`RPCTraceOpenTelemetry`的构造函数指定。代码参考：[src/module/rpc_trace_filter.h](https://github.com/sogou/srpc/blob/master/src/module/rpc_trace_filter.h#L238)
默认每秒收集1000条trace信息，并且透传tracing信息等其他功能也已遵循上述规范实现。
### 4. Attributes
我们可以通过`add_attributes()`添加某些额外的信息，比如数据规范中的OTEL_RESOURCE_ATTRIBUTES。
注意我们的service名"Example"也是设置到attributes中的，key为`service.name`。如果用户也在OTEL_RESOURCE_ATTRIBUTES中使用了`service.name`这个key，则SRPC的service name优先级更高，参考：[OpenTelemetry#resource](https://github.com/open-telemetry/opentelemetry-dotnet/tree/main/src/OpenTelemetry#resource)
### 5. Log和Baggage
SRPC提供了`log()`和`baggage()`接口，用户可以添加需要通过链路透传的数据。
```cpp
void log(const RPCLogVector& fields);
void baggage(const std::string& key, const std::string& value);
```
作为Server，我们可以使用`RPCContext`上的接口来添加log：
```cpp
class ExampleServiceImpl : public Example::Service
{
public:
void Echo(EchoRequest *req, EchoResponse *resp, RPCContext *ctx) override
{
resp->set_message("Hi back");
ctx->log({{"event", "info"}, {"message", "rpc server echo() end."}});
}
};
```
作为client，我们可以使用`RPCClientTask`上的接口添加log：
```cpp
srpc::SRPCClientTask *task = client.create_Echo_task(...);
task->log({{"event", "info"}, {"message", "log by rpc client echo()."}});
```
sogou-srpc-ac65014/docs/docs-09-metrics.md 0000664 0000000 0000000 00000026411 15252174244 0020246 0 ustar 00root root 0000000 0000000 [English version](/docs/en/docs-09-metrics.md)
## 09 - 上报Metrics
**Metrics**(指标)是常用的监控需求，**SRPC**支持产生与统计Metrics，并通过多种途径上报，其中包括上报到[Prometheus](https://prometheus.io/)和[OpenTelemetry](https://opentelemetry.io)。
秉承着**Workflow**的风格，所有的上报都是异步任务推送或拉取模式，对RPC的请求和服务不会产生任何性能影响。
本文档会介绍Metrics的概念、结合tutorial-16对接口进行讲解、上报Prometheus的特点、上报OpenTelemetry的特点、以及介绍使用了thread local进行性能提速的Var模块。
### 1. Metrics概念介绍
**Prometheus**的数据类型可以参考官方文档：[Concepts - Metrics](https://prometheus.io/docs/concepts/metric_types/) 和 [Type of Metrics](https://prometheus.io/docs/tutorials/understanding_metric_types/)。
**OpenTelemetry**的可以参考[数据规范(data specification)](https://github.com/open-telemetry/opentelemetry-specification)以及[Metrics的datamodel.md](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/datamodel.md)。
其中四种基本指标的概念都是一致的，因此SRPC对这四种指标进行了最基本的支持，可分别对应到Prometheus和OpenTelemetry的上报数据中。参考下表：
|指标类型|Prometheus|OpenTelemetry| SRPC中的创建接口 | SRPC中的常用操作 |
|-------|----------|------------|-----------------|-------------------|
|单个数值| Gauge | Gauge | create_gauge(name, help); | void increase(); void decrease();|
|计数器 | Counter | Sum | create_counter(name, help); |GaugeVar *add(labels);|
|直方图 |Histogram | Histogram | create_histogram(name, help, buckets); |void observe(data);|
|采样 | Summary | Summary | create_summary(name, help, quantiles); |void observe(data);|
四种指标的大致描述：
1. **单个数值**：简单数值的度量，可以简单的增减。
2. **计数器**：累计的度量，可以通过添加若干**label**去区分不同label下的数值。
3. **直方图**：对观察observer()得到的接口进行分区间累加，因此需要传入**buckets**告知需要划分的区间；
比如传入的bucket分桶值是{ 1, 10, 100 }这3个数，则我们可以得到数据分布于{ 0 - 1, 0 - 10, 0 - 100, 0 - +Inf }4个区间的数据累计值，同时也会得到整体的总以及数据个数。
4. **采样**：与直方图类似，但传入的是**分位数quantiles**，比如{0.5, 0.9}以及它们的精度，主要用于事先不知道数据分布具体数值的场景（相比之下，直方图需要传入具体数值）。
采样是带有**时间窗口**的，可以通过接口指定**max_age统计窗口时长**以及**age_bucket内部分桶个数**，SRPC中默认的是60秒分5个桶切换时间窗口。
考虑到**采样Summary**本身的统计复杂性，一般来说建议优先使用**直方图Histogram**。
为了方便用户使用，目前SRPC里这四种指标都使用double类型进行统计。
### 2. 用法示例
我们结合[tutorial-16-server_with_metrics.cc](/tutorial/tutorial-16-server_with_metrics.cc)看看基本用法，本示例虽然是server，但client中的用法也是一样的。
#### (1) 创建插件
我们选择Prometheus作为我们的上报对象，因此需要使用**RPCMetricsPull**插件。
~~~cpp
#include "srpc/rpc_metrics_filter.h" // Metrics插件所在的头文件
int main()
{
SRPCServer server;
ExampleServiceImpl impl;
RPCMetricsPull filter; // 创建一个插件
filter.init(8080); // 配合Prometheus中填好的收集数据的端口
~~~
#### (2) 添加指标
这个**RPCMetricsPull**插件本身自带了统计部分常用指标，包括整体请求个数统计、按照service和method作为label的不同维度的请求个数统计以及请求耗时的分位数统计等。
用户可以自行增加想要统计的值，这里我们增加一个用于统计请求大小的直方图histogram，名字为"echo_request_size"，上报时的指标描述信息是"Echo request size"，数据bucket划分是 { 1, 10, 100 } 。
~~~cpp
filter.create_histogram("echo_request_size", "Echo request size", {1, 10, 100});
~~~
说明：
**添加指标的时机**
指标是可以随时添加的，即使server/client跑起来之后也可以。但**必须在操作这个指标之前添加**，否则获取指标的时候会获取到空指针，无法进行统计操作。
**指标的名字**
指标的名字是**全局唯一**的（无论是四种基本类型中的哪种），且**只能包含大小写字母和下划线，即a-z, A-Z, _** 。如果使用已经存在的名字创建指标，则会创建失败并返回NULL。
一旦创建成功，我们之后都会使用这同一个名字去操作这个指标。
**创建其他指标的接口**
可以查看刚才include的头文件[rpc_metrics_filter.h](/src/module/rpc_metrics_filter.h)中的：`class RPCMetricsFilter`。
#### (3) 把插件添加到server/client中
由于我们需要操作指标时，是需要调用这个插件上的接口的，因此我们在service中保留一下这个指针。这只是示例程序的用法，用户可以使用自己习惯的方式：
~~~cpp
class ExampleServiceImpl : public Example::Service
{
public:
void Echo(EchoRequest *req, EchoResponse *resp, RPCContext *ctx) override；
void set_filter(RPCMetricsPull *filter) { this->filter = filter; }
private:
RPCMetricsPull *filter; // 保留了插件的指针，并实现接口设置进去，这并非SRPC框架的接口
};
~~~
main函数中继续这样写：
~~~cpp
int main()
{
...
impl.set_filter(&filter); // 设置到我们刚才为service留的接口中
server.add_filter(&filter); // client也一样可以调用add_filter()
server.add_service(&impl);
filter.deinit();
return 0;
}
~~~
#### (4) 操作指标
我们在每次收到请求的时候，都把EchoRequest的大小统计到刚才创建的直方图指标上：
~~~cpp
class ExampleServiceImpl : public Example::Service
{
public:
void Echo(EchoRequest *req, EchoResponse *resp, RPCContext *ctx) override
{
resp->set_message("Hi back");
this->filter->histogram("echo_request_size")->observe(req->ByteSizeLong());
}
~~~
可以看到，我们通过filter上的histogram()接口，就可以带着刚才的名字去到指标的指针，并且通过observe()填入数据大小。
四种基本类型的获取接口如下：
~~~cpp
class RpcMetricsFilter : public RPCFilter
{
GaugeVar *gauge(const std::string& name);
CounterVar *counter(const std::string& name);
HistogramVar *histogram(const std::string& name);
SummaryVar *summary(const std::string& name);
~~~
如果找到，会返回四种基本指标类型的指针，可以如示例进行下一步操作比如histogram的统计接口observe()，**注意：不存在这个名字则会返回空指针，因此要保证我们拿到的变量一定是成功创建过的**。
四种类型常用的操作接口如上文表格所示，具体可以参考[rpc_var.h](/src/var/rpc_var.h)。
值得说明一下Counter类型的接口：
~~~cpp
class CounterVar : public RPCVar
{
GaugeVar *add(const std::map
We can find the span_id: **04d070f537f17d00** in client become parent_span_id: **04d070f537f17d00** in server:
### 2. Traces on Jaeger
Open the show page of Jaeger, we can find our service name **Example** and method name **Echo**. Here are two span nodes, which were reported by server and client respectively.
As what we saw on the screen, the client reported span_id: **04d070f537f17d00** and server reported span_id: **00202cf737f17d00**, these span and the correlated tracing information can be found on Jaeger, too.
### 3. About Parameters
How long to collect a trace, and the number of reported retries and other parameters can be specified through the constructor parameters of `RPCTraceOpenTelemetry`. Code reference: [src/module/rpc_trace_filter.h](https://github.com/sogou/srpc/blob/master/src/module/rpc_trace_filter.h#L238)
The default value is to collect up to 1000 trace information per second, and features such as transferring tracing information through the srpc framework transparently have also been implemented, which also conform to the specifications.
### 4. Attributes
We can also use `add_attributes()` to add some other informations as OTEL_RESOURCE_ATTRIBUTES.
Please notice that our service name "Example" is set also thought this attributes, the key of which is `service.name`. If `service.name` is also provided in OTEL_RESOURCE_ATTRIBUTES by users, then srpc service name takes precedence. Refers to : [OpenTelemetry#resource](https://github.com/open-telemetry/opentelemetry-dotnet/tree/main/src/OpenTelemetry#resource)
### 5. Log and Baggage
SRPC provides `log()` and `baggage()` to carry some user data through span.
API :
```cpp
void log(const RPCLogVector& fields);
void baggage(const std::string& key, const std::string& value);
```
As a server, we can use `RPCContext` to add log annotation:
```cpp
class ExampleServiceImpl : public Example::Service
{
public:
void Echo(EchoRequest *req, EchoResponse *resp, RPCContext *ctx) override
{
resp->set_message("Hi back");
ctx->log({{"event", "info"}, {"message", "rpc server echo() end."}});
}
};
```
As a client, we can use `RPCClientTask` to add log on span:
```cpp
srpc::SRPCClientTask *task = client.create_Echo_task(...);
task->log({{"event", "info"}, {"message", "log by rpc client echo()."}});
```
sogou-srpc-ac65014/docs/en/docs-09-metrics.md 0000664 0000000 0000000 00000027610 15252174244 0020652 0 ustar 00root root 0000000 0000000 [中文版](/docs/docs-09-metrics.md)
## 09 - Report Metrics to OpenTelemetry / Prometheus
**Metrics** are common monitoring requirements. **SRPC** supports the generation and statistics of Metrics, and reports through various way, including reporting to [Prometheus](https://prometheus.io/) and [OpenTelemetry](https://opentelemetry.io).
The report conforms to the **Workflow** style, which is pure asynchronous task or pull mode, and therefore has no performance impact on the RPC requests and services.
This document will introduce the concept of Metrics, explain the interface of [tutorial-16](/tutorial/tutorial-16-server_with_metrics.cc), the usage of reporting to Prometheus, the usage of reporting to OpenTelemetry, and introduce the Var module which uses thread local to speed up performance.
### 1. Introduction
The metrics type of **Prometheus** can be refered through the official documentation : [Concepts - Metrics](https://prometheus.io/docs/concepts/metric_types/) and [Type of Metrics](https://prometheus.io/docs/tutorials/understanding_metric_types/)。
The basic metrics concept of **OpenTelemetry** can be refer through [(data specification)](https://github.com/open-telemetry/opentelemetry-specification) and [Metrics的datamodel.md](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/datamodel.md).
The concepts of the four basic metrics are the same, so SRPC provides the most basic support for these four metrics, which can correspond to the reported data of Prometheus and OpenTelemetry respectively. Please refer to the table below:
|Metrics type|Prometheus|OpenTelemetry| create api in SRPC | some operations in SRPC |
|-------|----------|------------|-----------------|-------------------|
|single value| Gauge | Gauge | create_gauge(name, help); | void increase(); void decrease();|
|counter| Counter | Sum | create_counter(name, help); |GaugeVar *add(labels);|
|histogram |Histogram | Histogram | create_histogram(name, help, buckets); |void observe(data);|
|samples | Summary | Summary | create_summary(name, help, quantiles); |void observe(data);|
Basic description of the four types of metrics:
1. **Single value**: A simple value that can be increased or decreased.
2. **Counter**: A counter is a cumulative metrics, which can be added several **labels** to distinguish the values under different labels.
3. **Histogram**: A histogram samples observations (usually things like request durations or response sizes) and counts them in configurable buckets, so it is necessary to pass in **buckets** to inform the interval to be divided.
For example, if the input bucket value is { 1, 10, 100 }, then we can get the data distributed in these four intervals { 0 - 1, 0 - 10, 0 - 100, 0 - +Inf }. It also provides a sum and the total count of all observed values.
4. **Samples**: Similar to histogram, but **quantiles** are passed in, such as {0.5, 0.9} and their precision. It mainly used in scenarios where the specific value of the data distribution is not known in advance (In contrast, histograms require specific values to be passed in).
Sampling comes with a **time window**. You may specify the statistics window by **max_age** and the number of buckets by **age_bucket**. The default in SRPC is using 5 bucket in 60 seconds to switch times window.
Considering the statistical complexity of **sampling Summary**, **Histogram** is recommended in preference instead.
For convenience, these four type of metrics in SRPC are of type **double**.
### 2. Usage
The example shows usage via [tutorial-16-server_with_metrics.cc](/tutorial/tutorial-16-server_with_metrics.cc). Althought this example is for the server, the usage on the client is the same.
#### (1) create filter
Here we use Prometheus to be reported, thus we need the filter **RPCMetricsPull** as the plugin.
~~~cpp
#include "srpc/rpc_metrics_filter.h" // metrics filter header file
int main()
{
SRPCServer server;
ExampleServiceImpl impl;
RPCMetricsPull filter; // create a filter
filter.init(8080); // the port for Prometheus to pull metrics data
~~~
#### (2) add metrics
By default, there are some common metrics in the filter **RPCMetricsPull**, including the number of overall request, the number of requests in different service or method as labels, and the quantiles of latency.
Users can add any metrics they want. Here we add a histogram namee "echo_request_size" to count the size of requests. This metric describes the information as "Echo request size", and the data bucket division is { 1, 10, 100 }.
~~~cpp
filter.create_histogram("echo_request_size", "Echo request size", {1, 10, 100});
~~~
More illutrations:
**Timing to add metrics**
Metrics can be added any time, even after the server/client has been running. But **must be added before we use** this metrics, otherwise a NULL pointer will be obtained when we acquire the metrics by name.
**The name of the metrics**
The name of metrics is **globally unique** (no matter which of the four basic types), and **can only contain uppercase and lowercase letters and underscores, ie a~z, A~Z, _** . The creation will fail and return NULL if we create with an existed metric‘s name.
We will use this unique name to operate the metrics after it has been created.
**Other APIs to create metrics**
You can check this header file [rpc_metrics_filter.h](/src/module/rpc_metrics_filter.h): `class RPCMetricsFilter`.
#### (3) add filter into server / client
The following example keeps the pointer of this filter in ExampleServiceImpl because we may need the APIs on it to operate our metrics. It`s just one of the sample usages and users may use it in the way you used to.
~~~cpp
class ExampleServiceImpl : public Example::Service
{
public:
void Echo(EchoRequest *req, EchoResponse *resp, RPCContext *ctx) override；
void set_filter(RPCMetricsPull *filter) { this->filter = filter; }
private:
RPCMetricsPull *filter; // keep the pointer of filter and add set_filter(), notice that it's not APIs of SRPC
};
~~~
The usage in main:
~~~cpp
int main()
{
...
impl.set_filter(&filter); // keep the filter throught the APIs in service we mentioned above
server.add_filter(&filter); // client can also call add_filter()
server.add_service(&impl);
filter.deinit();
return 0;
}
~~~
#### (4) operation with the metrics
Every time we receive a request, we get cumulative calculation of the size of the EchoRequest with the histogram we just created.
~~~cpp
class ExampleServiceImpl : public Example::Service
{
public:
void Echo(EchoRequest *req, EchoResponse *resp, RPCContext *ctx) override
{
resp->set_message("Hi back");
this->filter->histogram("echo_request_size")->observe(req->ByteSizeLong());
}
~~~
As has been shown, we use the API `histogram()` on filter to get the histogram metrics by it's name. Then we use `observe()` to fill in the data size.
Let's take a look at the APIs to acquire four types of metrics.
~~~cpp
class RpcMetricsFilter : public RPCFilter
{
GaugeVar *gauge(const std::string& name);
CounterVar *counter(const std::string& name);
HistogramVar *histogram(const std::string& name);
SummaryVar *summary(const std::string& name);
~~~
If the metric is found by this name, the relevant type of pointer will be returned and we may use it to do the relevant next step, such as calling the observe() of historgram we did above. **Notice : If the metric name does not exist, Null will be returned, so we must ensure that the metrics we want must have be successfully created.**
Common APIs can be refered to the table above. For more details please refer to [rpc_var.h](/src/var/rpc_var.h).
It's worth taking a look at the API on Counter:
~~~cpp
class CounterVar : public RPCVar
{
GaugeVar *add(const std::map88<4"sQ XDh
ٞ} 2gc xN76)/;B40p,xV3