SkyWalking-OAL-使用教程

简介

说明

本文介绍SkyWalking的OAL语法的用法。

本文只介绍OAL，OAL要与告警结合起来才有意义，告警见此文：SkyWalking-告警-使用教程 – 自学精灵

官网

OAL介绍

https://github.com/apache/skywalking/blob/master/docs/en/guides/backend-oal-scripts.md

OAL规则语法：https://github.com/apache/skywalking/blob/master/docs/en/concepts-and-designs/oal.md

范围和字段：https://github.com/apache/skywalking/blob/master/docs/en/concepts-and-designs/scope-definitions.md

OAL简介

SkyWalking从8.0.0开始支持OAL脚本，它所在路径为：config/oal/*.oal。我们可以修改它，比如：添加过滤条件或者新的衡量标准，重启OAP生效。

Apache SkyWalking告警是由一组规则驱动，这些规则定义在config/alarm-settings.yml文件中，alarm-settings.yml中的rules.xxx_rule.metrics-name对应的是config/oal路径下的配置文件中的详细规则：core.oal、event.oal，java-agent.oal， browser.oal。

endpoint 规则相比 service、instance 规则耗费更多内存及资源。

OAL(Observability Analysis Language）：观测分析语言。

在流模式(Streaming mode)下，SkyWalking 提供了OAL来分析流入的数据。OAL 聚焦于服务，服务实例以及端点的度量指标，因此 OAL 非常易于学习和使用。

6.3版本以后，OAL引擎嵌入在OAP服务器运行时中，称为oal-rt(OAL运行时)。OAL脚本现在位于/config文件夹，用户可以简单地改变和重新启动服务器，使其有效。

但是，OAL脚本仍然是编译语言，OAL运行时动态生成Java代码。您可以在系统环境上设置SW_OAL_ENGINE_DEBUG=Y，查看生成了哪些类。

配置示例

// 计算Endpoint1 和 Endpoint2 的 p99。
endpoint_p99 = from(Endpoint.latency).filter(name in (“Endpoint1”, “Endpoint2”)).summary(0.99)

// 计算以“serv”开头的端点名字的 p99。
serv_Endpoint_p99 = from(Endpoint.latency).filter(name like “serv%”).summary(0.99)

// 计算每个端点的响应平均时长
endpoint_avg = from(Endpoint.latency).avg()

// 计算每个端点 p50，p75，p90，p95 and p99 的延迟柱状图，每隔 50 毫秒一条柱
endpoint_percentile = from(Endpoint.latency).percentile(10)

// 统计每个服务响应状态为 true 的百分比
endpoint_success = from(Endpoint.*).filter(status == true).percent()

// 计算每个服务的响应码为[404, 500, 503]的总和
endpoint_abnormal = from(Endpoint.*).filter(responseCode in [404, 500, 503]).count()

// 计算每个服务的请求类型为[PRC, gRPC]的总和
endpoint_rpc_calls_sum = from(Endpoint.*).filter(type in [RequestType.PRC, RequestType.gRPC]).sum()

// 计算每个端点的端点名称为[“/v1”, “/v2”]的总和
endpoint_url_sum = from(Endpoint.*).filter(endpointName in [“/v1”, “/v2”]).sum()

// 统计每个服务的调用总量
endpoint_calls = from(Endpoint.*).count()

// 计算每个服务的GET方法的CPM。值的组成为：`tagKey:tagValue`.
// 方案1, 使用`tags contain`.
service_cpm_http_get = from(Service.*).filter(tags contain “http.method:GET”).cpm()
// 方案2, 使用 `tag[key]`.
service_cpm_http_get = from(Service.*).filter(tag[“http.method”] == “GET”).cpm();

// 计算每个服务的除了GET的方法的CPM。值的组成为：`tagKey:tagValue`.
service_cpm_http_other = from(Service.*).filter(tags not contain “http.method:GET”).cpm()

// 计算浏览应用的错误率。分子是FIRST_ERROR，分母是NORMAL
browser_app_error_rate = from(BrowserAppTraffic.*).rate(trafficCategory == BrowserAppTrafficCategory.FIRST_ERROR, trafficCategory == BrowserAppTrafficCategory.NORMAL);

disable(segment);
disable(endpoint_relation_server_side);
disable(top_n_database_statement);

默认的配置

config/oal/core.oal

/*
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the "License"); you may not use this file except in compliance with
 * the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 *
 */

// For services using protocols HTTP 1/2, gRPC, RPC, etc., the cpm metrics means "calls per minute",
// for services that are built on top of TCP, the cpm means "packages per minute".

// All scope metrics
all_percentile = from(All.latency).percentile(10);  // Multiple values including p50, p75, p90, p95, p99
all_heatmap = from(All.latency).histogram(100, 20);

// Service scope metrics
service_resp_time = from(Service.latency).longAvg();
service_sla = from(Service.*).percent(status == true);
service_cpm = from(Service.*).cpm();
service_percentile = from(Service.latency).percentile(10); // Multiple values including p50, p75, p90, p95, p99
service_apdex = from(Service.latency).apdex(name, status);

// Service relation scope metrics for topology
service_relation_client_cpm = from(ServiceRelation.*).filter(detectPoint == DetectPoint.CLIENT).cpm();
service_relation_server_cpm = from(ServiceRelation.*).filter(detectPoint == DetectPoint.SERVER).cpm();
service_relation_client_call_sla = from(ServiceRelation.*).filter(detectPoint == DetectPoint.CLIENT).percent(status == true);
service_relation_server_call_sla = from(ServiceRelation.*).filter(detectPoint == DetectPoint.SERVER).percent(status == true);
service_relation_client_resp_time = from(ServiceRelation.latency).filter(detectPoint == DetectPoint.CLIENT).longAvg();
service_relation_server_resp_time = from(ServiceRelation.latency).filter(detectPoint == DetectPoint.SERVER).longAvg();
service_relation_client_percentile = from(ServiceRelation.latency).filter(detectPoint == DetectPoint.CLIENT).percentile(10); // Multiple values including p50, p75, p90, p95, p99
service_relation_server_percentile = from(ServiceRelation.latency).filter(detectPoint == DetectPoint.SERVER).percentile(10); // Multiple values including p50, p75, p90, p95, p99

// Service Instance relation scope metrics for topology
service_instance_relation_client_cpm = from(ServiceInstanceRelation.*).filter(detectPoint == DetectPoint.CLIENT).cpm();
service_instance_relation_server_cpm = from(ServiceInstanceRelation.*).filter(detectPoint == DetectPoint.SERVER).cpm();
service_instance_relation_client_call_sla = from(ServiceInstanceRelation.*).filter(detectPoint == DetectPoint.CLIENT).percent(status == true);
service_instance_relation_server_call_sla = from(ServiceInstanceRelation.*).filter(detectPoint == DetectPoint.SERVER).percent(status == true);
service_instance_relation_client_resp_time = from(ServiceInstanceRelation.latency).filter(detectPoint == DetectPoint.CLIENT).longAvg();
service_instance_relation_server_resp_time = from(ServiceInstanceRelation.latency).filter(detectPoint == DetectPoint.SERVER).longAvg();
service_instance_relation_client_percentile = from(ServiceInstanceRelation.latency).filter(detectPoint == DetectPoint.CLIENT).percentile(10); // Multiple values including p50, p75, p90, p95, p99
service_instance_relation_server_percentile = from(ServiceInstanceRelation.latency).filter(detectPoint == DetectPoint.SERVER).percentile(10); // Multiple values including p50, p75, p90, p95, p99

// Service Instance Scope metrics
service_instance_sla = from(ServiceInstance.*).percent(status == true);
service_instance_resp_time= from(ServiceInstance.latency).longAvg();
service_instance_cpm = from(ServiceInstance.*).cpm();

// Endpoint scope metrics
endpoint_cpm = from(Endpoint.*).cpm();
endpoint_avg = from(Endpoint.latency).longAvg();
endpoint_sla = from(Endpoint.*).percent(status == true);
endpoint_percentile = from(Endpoint.latency).percentile(10); // Multiple values including p50, p75, p90, p95, p99

// Endpoint relation scope metrics
endpoint_relation_cpm = from(EndpointRelation.*).filter(detectPoint == DetectPoint.SERVER).cpm();
endpoint_relation_resp_time = from(EndpointRelation.rpcLatency).filter(detectPoint == DetectPoint.SERVER).longAvg();
endpoint_relation_sla = from(EndpointRelation.*).filter(detectPoint == DetectPoint.SERVER).percent(status == true);
endpoint_relation_percentile = from(EndpointRelation.rpcLatency).filter(detectPoint == DetectPoint.SERVER).percentile(10); // Multiple values including p50, p75, p90, p95, p99

database_access_resp_time = from(DatabaseAccess.latency).longAvg();
database_access_sla = from(DatabaseAccess.*).percent(status == true);
database_access_cpm = from(DatabaseAccess.*).cpm();
database_access_percentile = from(DatabaseAccess.latency).percentile(10);

config/oal/event.oal 

/*
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the "License"); you may not use this file except in compliance with
 * the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 *
 */

event_total = from(Event.*).count();

event_normal_count = from(Event.*).filter(type == "Normal").count();
event_error_count = from(Event.*).filter(type == "Error").count();

event_start_count = from(Event.*).filter(name == "Start").count();
event_shutdown_count = from(Event.*).filter(name == "Shutdown").count();

config/oal/java-agent.oal 

/*
 * Licensed to the Apache Software Foundation (ASF) under one or more
 * contributor license agreements.  See the NOTICE file distributed with
 * this work for additional information regarding copyright ownership.
 * The ASF licenses this file to You under the Apache License, Version 2.0
 * (the "License"); you may not use this file except in compliance with
 * the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 *
 */

// JVM instance metrics
instance_jvm_cpu = from(ServiceInstanceJVMCPU.usePercent).doubleAvg();
instance_jvm_memory_heap = from(ServiceInstanceJVMMemory.used).filter(heapStatus == true).longAvg();
instance_jvm_memory_noheap = from(ServiceInstanceJVMMemory.used).filter(heapStatus == false).longAvg();
instance_jvm_memory_heap_max = from(ServiceInstanceJVMMemory.max).filter(heapStatus == true).longAvg();
instance_jvm_memory_noheap_max = from(ServiceInstanceJVMMemory.max).filter(heapStatus == false).longAvg();
instance_jvm_young_gc_time = from(ServiceInstanceJVMGC.time).filter(phrase == GCPhrase.NEW).sum();
instance_jvm_old_gc_time = from(ServiceInstanceJVMGC.time).filter(phrase == GCPhrase.OLD).sum();
instance_jvm_young_gc_count = from(ServiceInstanceJVMGC.count).filter(phrase == GCPhrase.NEW).sum();
instance_jvm_old_gc_count = from(ServiceInstanceJVMGC.count).filter(phrase == GCPhrase.OLD).sum();
instance_jvm_thread_live_count = from(ServiceInstanceJVMThread.liveCount).longAvg();
instance_jvm_thread_daemon_count = from(ServiceInstanceJVMThread.daemonCount).longAvg();
instance_jvm_thread_peak_count = from(ServiceInstanceJVMThread.peakCount).longAvg();
instance_jvm_thread_runnable_state_thread_count = from(ServiceInstanceJVMThread.runnableStateThreadCount).longAvg();
instance_jvm_thread_blocked_state_thread_count = from(ServiceInstanceJVMThread.blockedStateThreadCount).longAvg();
instance_jvm_thread_waiting_state_thread_count = from(ServiceInstanceJVMThread.waitingStateThreadCount).longAvg();
instance_jvm_thread_timed_waiting_state_thread_count = from(ServiceInstanceJVMThread.timedWaitingStateThreadCount).longAvg();
instance_jvm_class_loaded_class_count = from(ServiceInstanceJVMClass.loadedClassCount).longAvg();
instance_jvm_class_total_unloaded_class_count = from(ServiceInstanceJVMClass.totalUnloadedClassCount).longAvg();
instance_jvm_class_total_loaded_class_count = from(ServiceInstanceJVMClass.totalLoadedClassCount).longAvg();

config/oal/browser.oal 

/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements.  See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License.  You may obtain a copy of the License at
*
*     http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
// browser app
browser_app_pv = from(BrowserAppTraffic.count).filter(trafficCategory == BrowserAppTrafficCategory.NORMAL).sum();
browser_app_error_rate = from(BrowserAppTraffic.*).rate(trafficCategory == BrowserAppTrafficCategory.FIRST_ERROR,trafficCategory == BrowserAppTrafficCategory.NORMAL);
browser_app_error_sum = from(BrowserAppTraffic.count).filter(trafficCategory != BrowserAppTrafficCategory.NORMAL).sum();

// browser app single version
browser_app_single_version_pv = from(BrowserAppSingleVersionTraffic.count).filter(trafficCategory == BrowserAppTrafficCategory.NORMAL).sum();
browser_app_single_version_error_rate = from(BrowserAppSingleVersionTraffic.trafficCategory).rate(trafficCategory == BrowserAppTrafficCategory.FIRST_ERROR,trafficCategory == BrowserAppTrafficCategory.NORMAL);
browser_app_single_version_error_sum = from(BrowserAppSingleVersionTraffic.count).filter(trafficCategory != BrowserAppTrafficCategory.NORMAL).sum();

// browser app page
browser_app_page_pv = from(BrowserAppPageTraffic.count).filter(trafficCategory == BrowserAppTrafficCategory.NORMAL).sum();
browser_app_page_error_rate = from(BrowserAppPageTraffic.*).rate(trafficCategory == BrowserAppTrafficCategory.FIRST_ERROR,trafficCategory == BrowserAppTrafficCategory.NORMAL);
browser_app_page_error_sum = from(BrowserAppPageTraffic.count).filter(trafficCategory != BrowserAppTrafficCategory.NORMAL).sum();

browser_app_page_ajax_error_sum = from(BrowserAppPageTraffic.count).filter(trafficCategory != BrowserAppTrafficCategory.NORMAL).filter(errorCategory == BrowserErrorCategory.AJAX).sum();
browser_app_page_resource_error_sum = from(BrowserAppPageTraffic.count).filter(trafficCategory != BrowserAppTrafficCategory.NORMAL).filter(errorCategory == BrowserErrorCategory.RESOURCE).sum();
browser_app_page_js_error_sum = from(BrowserAppPageTraffic.count).filter(trafficCategory != BrowserAppTrafficCategory.NORMAL).filter(errorCategory in [BrowserErrorCategory.JS,BrowserErrorCategory.VUE,BrowserErrorCategory.PROMISE]).sum();
browser_app_page_unknown_error_sum = from(BrowserAppPageTraffic.count).filter(trafficCategory != BrowserAppTrafficCategory.NORMAL).filter(errorCategory == BrowserErrorCategory.UNKNOWN).sum();

// browser performance metrics
browser_app_page_redirect_avg = from(BrowserAppPagePerf.redirectTime).longAvg();
browser_app_page_dns_avg = from(BrowserAppPagePerf.dnsTime).longAvg();
browser_app_page_ttfb_avg = from(BrowserAppPagePerf.ttfbTime).longAvg();
browser_app_page_tcp_avg = from(BrowserAppPagePerf.tcpTime).longAvg();
browser_app_page_trans_avg = from(BrowserAppPagePerf.transTime).longAvg();
browser_app_page_dom_analysis_avg = from(BrowserAppPagePerf.domAnalysisTime).longAvg();
browser_app_page_fpt_avg = from(BrowserAppPagePerf.fptTime).longAvg();
browser_app_page_dom_ready_avg = from(BrowserAppPagePerf.domReadyTime).longAvg();
browser_app_page_load_page_avg = from(BrowserAppPagePerf.loadPageTime).longAvg();
browser_app_page_res_avg = from(BrowserAppPagePerf.resTime).longAvg();
browser_app_page_ssl_avg = from(BrowserAppPagePerf.sslTime).longAvg();
browser_app_page_ttl_avg = from(BrowserAppPagePerf.ttlTime).longAvg();
browser_app_page_first_pack_avg = from(BrowserAppPagePerf.firstPackTime).longAvg();
browser_app_page_fmp_avg = from(BrowserAppPagePerf.fmpTime).longAvg();

browser_app_page_fpt_percentile = from(BrowserAppPagePerf.fptTime).percentile(10);
browser_app_page_ttl_percentile = from(BrowserAppPagePerf.ttlTime).percentile(10);
browser_app_page_dom_ready_percentile = from(BrowserAppPagePerf.domReadyTime).percentile(10);
browser_app_page_load_page_percentile = from(BrowserAppPagePerf.loadPageTime).percentile(10);
browser_app_page_first_pack_percentile = from(BrowserAppPagePerf.firstPackTime).percentile(10);
browser_app_page_fmp_percentile = from(BrowserAppPagePerf.fmpTime).percentile(10);

// Disable unnecessary hard core stream, targeting @Stream#name
/////////
//disable(browser_error_log);

OAL语法

OAL 脚本文件应该以 .oal 为后缀。

// Declare the metrics.
METRICS_NAME = from(SCOPE.(* | [FIELD][,FIELD ...]))
[.filter(FIELD OP [INT | STRING])]
.FUNCTION([PARAM][, PARAM ...])

// Disable hard code 
disable(METRICS_NAME);

域(Scope)

域包括全局(All)、服务(Service)、服务实例(Service Instance)、端点(Endpoint)、服务关系(Service Relation)、服务实例关系(Service Instance Relation)、端点关系(Endpoint Relation)。

当然还有一些字段，他们都属于以上某个域。

过滤器(Filter)

使用在使用过滤器的时候，通过指定字段名或表达式来构建字段值的过滤条件。

表达式可以使用 and，or 和 () 进行组合。

操作符包含==，!=，>，<，>=，<=，in […]，like %…，like …%，like %…%，他们可以基于字段类型进行类型检测,

如果类型不兼容会在编译/代码生成期间报错。

聚合函数(Aggregation Function)

默认的聚合函数由 SkyWalking OAP 核心实现。并可自由扩展更多函数。

提供的函数：

longAvg：某个域实体所有输入的平均值，输入字段必须是 long 类型。

instance_jvm_memory_max = from(ServiceInstanceJVMMemory.max).longAvg();

在上面的例子中，输入是 ServiceInstanceJVMMemory 域的每个请求，平均值是基于字段 max 进行求值的。

doubleAvg：某个域实体的所有输入的平均值，输入的字段必须是 double 类型。

​​​​​​​instance_jvm_cpu = from(ServiceInstanceJVMCPU.usePercent).doubleAvg();

在上面的例子中，输入是 ServiceInstanceJVMCPU 域的每个请求，平均值是基于 usePercent 字段进行求值的。

percent：对于输入中匹配指定条件的百分比数.

endpoint_percent = from(Endpoint.*).percent(status == true);

在上面的例子中，输入是每个端点的请求，条件是 endpoint.status == true。

rate：对于条件匹配的输入，比率以100的分数表示。

​​​​​​​browser_app_error_rate = from(BrowserAppTraffic.*).rate(trafficCategory == BrowserAppTrafficCategory.FIRST_ERROR, trafficCategory == BrowserAppTrafficCategory.NORMAL);

在上面的例子中，所有的输入都是每个浏览器应用流量的请求。分子的条件是trafficCategory == BrowserAppTrafficCategory.FIRST_ERROR，分母的条件是trafficCategory == BrowserAppTrafficCategory.NORMAL。

其中，第一个参数是分子的条件，第二个参数是分母的条件。

sum：某个域实体的调用总数。

​​​​​​​service_calls_sum = from(Service.*).sum();

在上面的例子中，统计每个服务的调用数。

histogram：热力图   更多详见Heatmap in WIKI。

all_heatmap = from(All.latency).histogram(100, 20);

在上面的例子中，计算了所有传入请求的热力学热图。

第一个参数是计算延迟的精度，在上面的例子中，在101-200ms组中，113ms和193ms被认为是相同的。

第二个参数是分组数量，在上面的例子中，一共有21组数据分别为0-100ms，101-200ms……1901-2000ms，2000ms以上.

apdex：应用性能指数(Application Performance Index)

service_apdex = from(Service.latency).apdex(name, status);

在上面的例子中，计算了所有服务的应用性能指数。

第一个参数是服务名称，该名称的Apdex阈值在配置文件service-apdex-threshold.yml中定义。

第二个参数是请求状态，状态(成功或失败)影响Apdex的计算。

P99，P95，P90，P75，P50：百分位  更多详见Percentile in WIKI

百分位是自7.0版本引入的第一个多值度量。由于有多个值，可以通过getMultipleLinearIntValuesGraphQL查询进行查询。

all_percentile = from(All.latency).percentile(10);

在上面的例子中，计算了所有传入请求的 P99，P95，P90，P75，P50。参数是百分位计算的精度，在上例中120ms和124被认为是相同的。

度量指标名称(Metrics Name)

存储实现，告警以及查询模块的度量指标名称，SkyWalking 内核支持自动类型推断。

组(Group)

所有度量指标数据都会使用 Scope.ID 和最小时间桶(min-level time bucket) 进行分组。

在端点的域中，Scope.ID 为端点的 ID（基于服务及其端点的唯一标志）。

强制转换(Cast)

源的字段是静态类型。在一些情况下，过滤语句和聚合语句所需要的字段类型和源的字段类型不匹配，例如：源的tag的值是String类型，大部分的聚合计算需要是数字类型。强制转换表达式就是用来解决这个的。

用法

• (str->long) or (long), cast string type into long.
• (str->int) or (int), cast string type into int.

示例：

mq_consume_latency = from((str->long)Service.tag["transmission.latency"]).longAvg(); // the value of tag is string type.

强制转换表达式支持如下位置：

• From statement. from((cast)source.attre).
• Filter expression. .filter((cast)tag[“transmission.latency”] > 0)
• Aggregation function parameter. .longAvg((cast)strField1== 1, (cast)strField2) 

禁用(Disable)

Disable是OAL中的高级语句，只在特定情况下使用。

一些聚合和度量是通过核心硬代码定义的，这个Disable语句是设计用来让它们停止活动的，
比如segment, top_n_database_statement。

在默认情况下，没有被禁用的。 ​