SpringBoot actuator监控定时任务执行时间

SpringBoot actuator监控定时任务执行时间

为了能够可视化任务执行情况，我们需要记录相应的度量指标

初始化项目

我们通过start.spring.io来快速生成Spring项目，添加Spring Web和Spring Boot Actuator，点击生成

我们来编写测试的定时任务，编写后启动，发现我们的job已正常执行

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
package com.bohanzhang.actuatordemo;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.scheduling.annotation.EnableScheduling;
import org.springframework.scheduling.annotation.Scheduled;

@SpringBootApplication
@EnableScheduling
public class ActuatordemoApplication {

	public static void main(String[] args) {
		SpringApplication.run(ActuatordemoApplication.class, args);
	}

	/**
	 * 5秒执行一次
	 * @throws InterruptedException
	 */
	@Scheduled(fixedRate = 5000)
	public void jobDemo() throws InterruptedException {
		System.out.println("start jobDemo");
		Thread.sleep(3000);
		System.out.println("end jobDemo");
	}
}

日志如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.2.5.RELEASE)

2020-03-05 20:58:18.180  INFO 17766 --- [           main] c.b.a.ActuatordemoApplication            : Starting ActuatordemoApplication on zhangbohans-MacBook-Pro.local with PID 17766 (/Users/bohan/Downloads/actuatordemo/target/classes started by bohan in /Users/bohan/Downloads/actuatordemo)
2020-03-05 20:58:18.181  INFO 17766 --- [           main] c.b.a.ActuatordemoApplication            : No active profile set, falling back to default profiles: default
2020-03-05 20:58:18.794  INFO 17766 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8080 (http)
2020-03-05 20:58:18.799  INFO 17766 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2020-03-05 20:58:18.799  INFO 17766 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.31]
2020-03-05 20:58:18.842  INFO 17766 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2020-03-05 20:58:18.842  INFO 17766 --- [           main] o.s.web.context.ContextLoader            : Root WebApplicationContext: initialization completed in 637 ms
2020-03-05 20:58:19.046  INFO 17766 --- [           main] o.s.s.concurrent.ThreadPoolTaskExecutor  : Initializing ExecutorService 'applicationTaskExecutor'
2020-03-05 20:58:19.191  INFO 17766 --- [           main] o.s.s.c.ThreadPoolTaskScheduler          : Initializing ExecutorService 'taskScheduler'
2020-03-05 20:58:19.195  INFO 17766 --- [           main] o.s.b.a.e.web.EndpointLinksResolver      : Exposing 13 endpoint(s) beneath base path '/actuator'
start jobDemo
2020-03-05 20:58:19.271  INFO 17766 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
2020-03-05 20:58:19.274  INFO 17766 --- [           main] c.b.a.ActuatordemoApplication            : Started ActuatordemoApplication in 1.284 seconds (JVM running for 1.601)
end jobDemo
start jobDemo
end jobDemo

下面我们来为这个job添加监控，监控主要依赖MeterRegistry来进行

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
package com.bohanzhang.actuatordemo;

import io.micrometer.core.instrument.MeterRegistry;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.scheduling.annotation.EnableScheduling;
import org.springframework.scheduling.annotation.Scheduled;

@SpringBootApplication
@EnableScheduling
public class ActuatordemoApplication {
	
	private MeterRegistry meterRegistry;

	public ActuatordemoApplication(MeterRegistry meterRegistry) {
		this.meterRegistry = meterRegistry;
	}

	public static void main(String[] args) {
		SpringApplication.run(ActuatordemoApplication.class, args);
	}

	/**
	 * 5秒执行一次
	 * @throws InterruptedException
	 */
	@Scheduled(fixedRate = 5000)
	public void jobDemo() throws InterruptedException {
		meterRegistry.timer("job.demo").record(() -> {
			System.out.println("start jobDemo");
			try {
				Thread.sleep(3000);
			} catch (InterruptedException e) {
				e.printStackTrace();
			}
			System.out.println("end jobDemo");
		});
		
	}
}

记录后如果查看呢？我们来访问/actuator，默认只打开了health,info，

1
2
$ curl http://localhost:8080/actuator
{"_links":{"self":{"href":"http://localhost:8080/actuator","templated":false},"health-path":{"href":"http://localhost:8080/actuator/health/{*path}","templated":true},"health":{"href":"http://localhost:8080/actuator/health","templated":false},"info":{"href":"http://localhost:8080/actuator/info","templated":false}}}%

我们可以在通过修改application.properties来展示所有的功能

1
management.endpoints.web.exposure.include=*

再次访问/actuator

1
2
curl http://localhost:8080/actuator
{"_links":{"self":{"href":"http://localhost:8080/actuator","templated":false},"beans":{"href":"http://localhost:8080/actuator/beans","templated":false},"caches-cache":{"href":"http://localhost:8080/actuator/caches/{cache}","templated":true},"caches":{"href":"http://localhost:8080/actuator/caches","templated":false},"health":{"href":"http://localhost:8080/actuator/health","templated":false},"health-path":{"href":"http://localhost:8080/actuator/health/{*path}","templated":true},"info":{"href":"http://localhost:8080/actuator/info","templated":false},"conditions":{"href":"http://localhost:8080/actuator/conditions","templated":false},"configprops":{"href":"http://localhost:8080/actuator/configprops","templated":false},"env":{"href":"http://localhost:8080/actuator/env","templated":false},"env-toMatch":{"href":"http://localhost:8080/actuator/env/{toMatch}","templated":true},"loggers-name":{"href":"http://localhost:8080/actuator/loggers/{name}","templated":true},"loggers":{"href":"http://localhost:8080/actuator/loggers","templated":false},"heapdump":{"href":"http://localhost:8080/actuator/heapdump","templated":false},"threaddump":{"href":"http://localhost:8080/actuator/threaddump","templated":false},"metrics-requiredMetricName":{"href":"http://localhost:8080/actuator/metrics/{requiredMetricName}","templated":true},"metrics":{"href":"http://localhost:8080/actuator/metrics","templated":false},"scheduledtasks":{"href":"http://localhost:8080/actuator/scheduledtasks","templated":false},"mappings":{"href":"http://localhost:8080/actuator/mappings","templated":false}}}%

想看我们的监控信息可以通过/actuator/metrics/job.demo来查看

1
2
$ curl http://localhost:8080/actuator/metrics/job.demo
{"name":"job.demo","description":null,"baseUnit":"seconds","measurements":[{"statistic":"COUNT","value":21.0},{"statistic":"TOTAL_TIME","value":63.06746522},{"statistic":"MAX","value":3.005089397}],"availableTags":[]}%

这里可以看到我们任务的执行次数、总用时及当前用时等

数据库版本管理工具Liquibase

研发过程中经常涉及到数据库变更，对表结构的修复及对数据的修改，为了保证各环境都能正确的进行变更我们可能需要维护一个数据库升级文档来保存这些记录，有需要升级的环境按文档进行升级。

这样手工维护有几个缺点：

  1. 无法保证每个环境都按要求执行
  2. 遇到问题不一定有相对的回滚语句
  3. 无法自动化

为了解决这些问题，我们进行了一些调研，主要调研对象是Liquibase和Flyway，我们希望通过数据库版本管理工具实现以下几个目标：

  1. 数据库升级
  2. 数据库回滚
  3. 版本标记

调研过程中发现Flyway据库回滚功能是增值功能且实现逻辑是通过我们的升级脚本来进行“智能”降级，不符合我们目前的使用场景，Flyway相关的介绍可以看我早期的另一篇介绍：https://segmentfault.com/a/1190000000422670

Liquibase

Liquibase帮助团队跟踪、版本化及部署数据库架构和逻辑修改

安装

检查JRE

1
2
3
4
$ java -version
java version "1.8.0_231"
Java(TM) SE Runtime Environment (build 1.8.0_231-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.231-b11, mixed mode)

如果没有安装Java请自行安装

安装Liquibase

下载Liquibase-Version#-bin.tar.gz 文件，
解压压缩包，将目录添加到环境变量中

1
$ export PATH="/opt/liquibase-3.8.2:$PATH"

这个命令重新启动命令行就不会生效了，如果要保证一直可用需要将这个领先设置到.bashrc或者.zshrc中

通过运行帮助命令验证安装

1
2
3
4
5
6
7
8
9
$ liquibase --help
17:12:10.389 [main] DEBUG liquibase.resource.ClassLoaderResourceAccessor - Opening jar:file:/opt/liquibase-3.8.2/liquibase.jar!/liquibase.build.properties as liquibase.build.properties
Starting Liquibase at 星期三, 04 十二月 2019 17:12:10 CST (version 3.8.2 #26 built at Tue Nov 26 04:53:39 UTC 2019)


Usage: java -jar liquibase.jar [options] [command]

Standard Commands:
...

配置文件

下面是以mysql为例的配置文件

1
2
3
4
5
6
7
$ cat liquibase.properties
driver: com.mysql.cj.jdbc.Driver
classpath: ./mysql-connector-java-8.0.18.jar
url: jdbc:mysql://127.0.0.1/test
username: root
password: 123456
changeLogFile: myChangeLog.xml

数据库升级

创建myChangeLog.xml文件，这个文件用来记录升级记录升级信息，初始化的内容

1
2
3
4
5
6
7
8
9
10
$ cat myChangeLog.xml
<?xml version="1.0" encoding="UTF-8"?>

<databaseChangeLog
  xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
         http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.8.xsd">

</databaseChangeLog>

Liquibase支持通过SQL描述的方式来创建数据库

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
$ cat myChangeLog.xml
<?xml version="1.0" encoding="UTF-8"?>

<databaseChangeLog
  xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
         http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.8.xsd">

    <changeSet id="1.0" author="bohan">
        <sql>
        CREATE TABLE `deparment` (
        `id` int(11) unsigned NOT NULL AUTO_INCREMENT,
        `name` varchar(100) COLLATE utf8mb4_bin DEFAULT NULL,
        PRIMARY KEY (`id`)
        ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;
        </sql>
    </changeSet>
</databaseChangeLog>
$ liquibase update
Liquibase Community 3.8.2 by Datical
Liquibase: Update has been successful.

通过执行liquibase update进行升级，升级后的数据库如下，已经为我们创建了数据库，同时Liquibase生成了两个表用来管理数据库升级记录

1
2
3
4
5
6
7
8
9
$ mysql -h 127.0.0.1 -uroot -p123456 test -e "show tables;"
mysql: [Warning] Using a password on the command line interface can be insecure.
+-----------------------+
| Tables_in_test        |
+-----------------------+
| DATABASECHANGELOG     |
| DATABASECHANGELOGLOCK |
| deparment             |
+-----------------------+

继续执行升级

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
$ cat myChangeLog.xml
<?xml version="1.0" encoding="UTF-8"?>

<databaseChangeLog
  xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
         http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.8.xsd">

    <changeSet id="1.0" author="bohan">
        <sql>
        CREATE TABLE `deparment` (
        `id` int(11) unsigned NOT NULL AUTO_INCREMENT,
        `name` varchar(100) COLLATE utf8mb4_bin DEFAULT NULL,
        PRIMARY KEY (`id`)
        ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;
        </sql>
    </changeSet>
    <changeSet id="1.1" author="bohan">
        <sql>
        insert into deparment values(1, "test");
        </sql>
    </changeSet>
</databaseChangeLog>
$ liquibase update
Liquibase Community 3.8.2 by Datical
Liquibase: Update has been successful.
$ mysql -h 127.0.0.1 -uroot -p test -e "select * from deparment;"
Enter password:
+----+------+
| id | name |
+----+------+
|  1 | test |
+----+------+

数据如预期被添加

通过SQL文件

数据库变更也可以通过sql文件形式引用，避免myChangeLog.xml文件过大

1
2
3
<changeSet id="1.1" author="bohan">
    <sqlFile path="./update_deparment_name.sql"></sqlFile>
</changeSet>

数据库回滚

1
2
3
4
5
6
liquibase --help
Usage: java -jar liquibase.jar [options] [command]

Standard Commands:
 rollbackCount <value>          Rolls back the last <value> change sets
                                applied to the database

我们来执行rollbackCount进行回滚

1
2
3
4
5
$ liquibase rollbackCount 1
Liquibase Community 3.8.2 by Datical
Rolling Back Changeset:myChangeLog.xml::1.0::bohan
Unexpected error running Liquibase: No inverse to liquibase.change.core.RawSQLChange created
For more information, please use the --logLevel flag

提示没有回滚SQL，修改我们的myChangeLog.xml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
$ cat myChangeLog.xml
<?xml version="1.0" encoding="UTF-8"?>

<databaseChangeLog
  xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
         http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.8.xsd">

    <changeSet id="1.0" author="bohan">
        <sql>
        CREATE TABLE `deparment` (
        `id` int(11) unsigned NOT NULL AUTO_INCREMENT,
        `name` varchar(100) COLLATE utf8mb4_bin DEFAULT NULL,
        PRIMARY KEY (`id`)
        ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;
        </sql>
        <rollback>
        DROP TABLE deparment;
        </rollback>
    </changeSet>
    <changeSet id="1.1" author="bohan">
        <sql>
        insert into deparment values(1, "test");
        </sql>
        <rollback>
        DELETE FROM deparment WHERE id = 1;
        </rollback>
    </changeSet>
</databaseChangeLog>

执行回滚，发现已经没有新增的记录了

1
2
3
4
5
6
liquibase rollbackCount 1
Liquibase Community 3.8.2 by Datical
Rolling Back Changeset:myChangeLog.xml::1.1::bohan
Liquibase: Rollback has been successful.
$ mysql -h 127.0.0.1 -uroot -p test -e "select * from deparment;"
Enter password:

再次执行，数据库也如预期被删除

1
2
3
4
5
6
7
8
9
10
11
12
$ liquibase rollbackCount 1
Liquibase Community 3.8.2 by Datical
Rolling Back Changeset:myChangeLog.xml::1.0::bohan
Liquibase: Rollback has been successful.
$ mysql -h 127.0.0.1 -uroot -p test -e "show tables;"
Enter password:
+-----------------------+
| Tables_in_test        |
+-----------------------+
| DATABASECHANGELOG     |
| DATABASECHANGELOGLOCK |
+-----------------------+

版本标记

Liquibase提供了完善的标签功能，经过刚刚的回滚到上一次操作后我们目前只执行了ID为1.0的变更

1
2
3
4
5
6
7
mysql> select * from DATABASECHANGELOG;
+-----+--------+-----------------+---------------------+---------------+----------+------------------------------------+-------------+----------+------+-----------+----------+--------+---------------+
| ID  | AUTHOR | FILENAME        | DATEEXECUTED        | ORDEREXECUTED | EXECTYPE | MD5SUM                             | DESCRIPTION | COMMENTS | TAG  | LIQUIBASE | CONTEXTS | LABELS | DEPLOYMENT_ID |
+-----+--------+-----------------+---------------------+---------------+----------+------------------------------------+-------------+----------+------+-----------+----------+--------+---------------+
| 1.0 | bohan  | myChangeLog.xml | 2019-12-05 03:15:18 |             1 | EXECUTED | 8:fe52f094e795797c89459e8f22483482 | sql         |          | NULL | 3.8.2     | NULL     | NULL   | 5515718387    |
+-----+--------+-----------------+---------------------+---------------+----------+------------------------------------+-------------+----------+------+-----------+----------+--------+---------------+
1 row in set (0.00 sec)

在实际开发中，我们升级版本时通常会需要同时执行多个变更，如果变量存在问题需要回滚时按数量回滚就比较麻烦了，我们需要对我们的变更进行标签标记，下面可能用到的命令如下：

1
2
3
4
5
6
7
8
9
10
11
12
liquibase --help
11:21:31.994 [main] DEBUG liquibase.resource.ClassLoaderResourceAccessor - Opening jar:file:/opt/liquibase-3.8.2/liquibase.jar!/liquibase.build.properties as liquibase.build.properties
Starting Liquibase at 星期四, 05 十二月 2019 11:21:31 CST (version 3.8.2 #26 built at Tue Nov 26 04:53:39 UTC 2019)


Usage: java -jar liquibase.jar [options] [command]

Standard Commands:
 rollback <tag>                 Rolls back the database to the the state is was
Maintenance Commands
 tag <tag string>          'Tags' the current database state for future rollback
 tagExists <tag string>    Checks whether the given tag is already existing

针对当前数据库，我们通过liquibase tag进行打标签操作

1
2
3
4
$ liquibase tag v1.0
Liquibase Community 3.8.2 by Datical
Successfully tagged 'root@172.17.0.1@jdbc:mysql://127.0.0.1/test'
Liquibase command 'tag' was executed successfully.

查看记录发现ID为1.0的记录TAG中已设置为v1.0，符合我们的预期

1
2
3
4
5
6
7
mysql> select * from DATABASECHANGELOG;
+-----+--------+-----------------+---------------------+---------------+----------+------------------------------------+-------------+----------+------+-----------+----------+--------+---------------+
| ID  | AUTHOR | FILENAME        | DATEEXECUTED        | ORDEREXECUTED | EXECTYPE | MD5SUM                             | DESCRIPTION | COMMENTS | TAG  | LIQUIBASE | CONTEXTS | LABELS | DEPLOYMENT_ID |
+-----+--------+-----------------+---------------------+---------------+----------+------------------------------------+-------------+----------+------+-----------+----------+--------+---------------+
| 1.0 | bohan  | myChangeLog.xml | 2019-12-05 03:15:18 |             1 | EXECUTED | 8:fe52f094e795797c89459e8f22483482 | sql         |          | v1.0 | 3.8.2     | NULL     | NULL   | 5515718387    |
+-----+--------+-----------------+---------------------+---------------+----------+------------------------------------+-------------+----------+------+-----------+----------+--------+---------------+
1 row in set (0.00 sec)

执行更新后如果需要回滚通过liquibase rollback v1.0即可

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
$ liquibase update
Liquibase Community 3.8.2 by Datical
Liquibase: Update has been successful.

mysql> select * from DATABASECHANGELOG;
+-----+--------+-----------------+---------------------+---------------+----------+------------------------------------+-------------+----------+------+-----------+----------+--------+---------------+
| ID  | AUTHOR | FILENAME        | DATEEXECUTED        | ORDEREXECUTED | EXECTYPE | MD5SUM                             | DESCRIPTION | COMMENTS | TAG  | LIQUIBASE | CONTEXTS | LABELS | DEPLOYMENT_ID |
+-----+--------+-----------------+---------------------+---------------+----------+------------------------------------+-------------+----------+------+-----------+----------+--------+---------------+
| 1.0 | bohan  | myChangeLog.xml | 2019-12-05 03:15:18 |             1 | EXECUTED | 8:fe52f094e795797c89459e8f22483482 | sql         |          | v1.0 | 3.8.2     | NULL     | NULL   | 5515718387    |
| 1.1 | bohan  | myChangeLog.xml | 2019-12-05 03:28:06 |             2 | EXECUTED | 8:695a5ec0b2b3ddc4a9beeeca530adebc | sql         |          | NULL | 3.8.2     | NULL     | NULL   | 5516486105    |
+-----+--------+-----------------+---------------------+---------------+----------+------------------------------------+-------------+----------+------+-----------+----------+--------+---------------+
2 rows in set (0.00 sec)


$ liquibase rollback v1.0
Liquibase Community 3.8.2 by Datical
Rolling Back Changeset:myChangeLog.xml::1.1::bohan
Liquibase: Rollback has been successful.

mysql> select * from DATABASECHANGELOG;
+-----+--------+-----------------+---------------------+---------------+----------+------------------------------------+-------------+----------+------+-----------+----------+--------+---------------+
| ID  | AUTHOR | FILENAME        | DATEEXECUTED        | ORDEREXECUTED | EXECTYPE | MD5SUM                             | DESCRIPTION | COMMENTS | TAG  | LIQUIBASE | CONTEXTS | LABELS | DEPLOYMENT_ID |
+-----+--------+-----------------+---------------------+---------------+----------+------------------------------------+-------------+----------+------+-----------+----------+--------+---------------+
| 1.0 | bohan  | myChangeLog.xml | 2019-12-05 03:15:18 |             1 | EXECUTED | 8:fe52f094e795797c89459e8f22483482 | sql         |          | v1.0 | 3.8.2     | NULL     | NULL   | 5515718387    |
+-----+--------+-----------------+---------------------+---------------+----------+------------------------------------+-------------+----------+------+-----------+----------+--------+---------------+
1 row in set (0.00 sec)

接口测试框架Karate入门

接口测试框架Karate入门

有效的接口测试可以为我们的接口质量保驾护航。Karate的中文翻译是空手道，是一款接口测试框架，BDD类型，非程序员也能上手。

你能学到什么

你可以通过这篇教程学会如何通过jar包方式使用Karate进行接口测试

你需要什么

  • 大约10分钟
  • JDK 1.8+

环境准备

本篇教程通过命令行执行jar文件的方式来进行接口测试编写。

首先创建工作目录，进入这个目录

1
2
$ mkdir KarateTest
$ cd KarateTest

下载执行文件。

通过https://dl.bintray.com/ptrthomas/karate/下载最新jar包，下载后保存到项目根目录中，现在我们的项目有了第一个文件，结构如下：

1
2
3
4
5
$ tree
.
└── karate-0.9.4.jar

0 directories, 1 file

编写第一个接口用例文件HttpbinGet.feature，内容如下

1
2
3
4
5
6
Feature: Httbin.org get method test

   Scenario: Get should be ok
    Given url 'http://httpbin.org/get'
    When method get
    Then status 200

添加完用例后目录结构为

1
2
3
4
5
6
$ tree
.
├── HttpbinGet.feature
└── karate-0.9.4.jar

0 directories, 2 files

执行用例

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
$ java -jar karate-0.9.4.jar HttpbinGet.feature
23:07:19.174 [main] INFO  com.intuit.karate.Main - Karate version: 0.9.4
23:07:19.617 [ForkJoinPool-1-worker-1] WARN  com.intuit.karate - skipping bootstrap configuration: could not find or read file: classpath:karate-config.js
23:07:19.843 [ForkJoinPool-1-worker-1] DEBUG com.intuit.karate - request:
1 > GET http://httpbin.org/get
1 > Accept-Encoding: gzip,deflate
1 > Connection: Keep-Alive
1 > Host: httpbin.org
1 > User-Agent: Apache-HttpClient/4.5.5 (Java/1.8.0_231)

23:07:20.634 [ForkJoinPool-1-worker-1] DEBUG com.intuit.karate - response time in milliseconds: 788.74
1 < 200
1 < Access-Control-Allow-Credentials: true
1 < Access-Control-Allow-Origin: *
1 < Connection: keep-alive
1 < Content-Type: application/json
1 < Date: Sat, 23 Nov 2019 15:07:20 GMT
1 < Referrer-Policy: no-referrer-when-downgrade
1 < Server: nginx
1 < X-Content-Type-Options: nosniff
1 < X-Frame-Options: DENY
1 < X-XSS-Protection: 1; mode=block
{
  "args": {},
  "headers": {
    "Accept-Encoding": "gzip,deflate",
    "Host": "httpbin.org",
    "User-Agent": "Apache-HttpClient/4.5.5 (Java/1.8.0_231)"
  },
  "origin": "111.192.170.27, 111.192.170.27",
  "url": "https://httpbin.org/get"
}


23:07:20.684 [pool-1-thread-1] INFO  com.intuit.karate.Runner - <<pass>> feature 1 of 1: HttpbinGet.feature
---------------------------------------------------------
feature: HttpbinGet.feature
report: target/surefire-reports/HttpbinGet.json
scenarios:  1 | passed:  1 | failed:  0 | time: 1.0301
---------------------------------------------------------
Karate version: 0.9.4
======================================================
elapsed:   1.41 | threads:    1 | thread time: 1.03
features:     1 | ignored:    0 | efficiency: 0.73
scenarios:    1 | passed:     1 | failed: 0
======================================================

可视化报告地址：target/cucumber-html-reports/overview-features.html

How to Convert Python dict to object?

假如我们有如下的数据：

1
d = {'a': 1, 'b': {'c': 2}, 'd': ["hi", {'foo': "bar"}]}

假如我需要访问c需要通过d.get('b').get('c')

如果能直接通过d.b.c去访问就好了

实现如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
class Struct:
  '''The recursive class for building and representing objects with.'''
  def __init__(self, obj):
    for k, v in obj.iteritems():
      if isinstance(v, dict):
        setattr(self, k, Struct(v))
      else:
        setattr(self, k, v)
  def __getitem__(self, val):
    return self.__dict__[val]
  def __repr__(self):
    return '{%s}' % str(', '.join('%s : %s' % (k, repr(v)) for
      (k, v) in self.__dict__.iteritems()))

使用：

1
2
s = Struct(d)
s.b.c

提交代码时来张自拍

首先安装imagesnap，https://github.com/alexwilliamsca/imagesnap或者Mac系统下通过homebrew安装：

1
brew install imagesnap

新建~/.gitshots目录：

1
mkdir ~/.gitshots

添加下面的文件post-commit到你代码目录的git hooks中：

1
2
3
4
5
6
7
#!/usr/bin/env ruby
file="~/.gitshots/#{Time.now.to_i}.jpg"
unless File.directory?(File.expand_path("../../rebase-merge", __FILE__))
  puts "Taking capture into #{file}!"
  system "imagesnap -q -w 3 #{file} &"
end
exit 0

为这个文件添加可执行权限：

1
chmod +x .git/hooks/post-commit

钩子(hooks)是一些在”$GIT-DIR/hooks”目录的脚本, 在被特定的事件(certain points)触发后被调用。当”git init”命令被调用后, 一些非常有用的示例钩子文件(hooks)被拷到新仓库的hooks目录中; 但是在默认情况下这些钩子(hooks)是不生效的。 把这些钩子文件(hooks)的”.sample”文件名后缀去掉就可以使它们生效了。

这样就可以了！😎

你还可以把这些图片合成为视频：http://www.dayofthenewdan.com/projects/tlassemble

我录的视频样例：http://7d9pyw.com1.z0.glb.clouddn.com/2015-11-26T23:49:57.198500_test.mov

原文链接：Take a photo of yourself every time you commit

GitHook参考：http://gitbook.liuhui998.com/5_8.html

Redis set中sadd插入时间测试

今天老郑问我“一个redis的set里面，最多能放多少数据，而不会有显著的性能下降？”，由于Redis中读取
的时间复杂度为O(1)，增加的复杂度为O(n)，这个问题还是有测试的价值的。

所以我直接写了下面这个测试用例来测试这个问题：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import unittest

import redis
import time


class RedisTest(unittest.TestCase):
    def setUp(self):
        self.redis_client = redis.StrictRedis()

    def test_add_to_set(self):
        for i in range(1000000):
            start = time.time()
            self.redis_client.sadd('test_set', i)
            end = time.time()

1
2
3
4
5
6
7
8
9
10
11
12
13
14
> python -m unittest tests/test_redis.py

... 省略了好多好多行
index: 999994, time: 0.00010895729064941406
index: 999995, time: 0.00014710426330566406
index: 999996, time: 0.00015497207641601562
index: 999997, time: 0.00015592575073242188
index: 999998, time: 0.00017213821411132812
index: 999999, time: 0.00010395050048828125
.
----------------------------------------------------------------------
Ran 1 test in 191.684s

OK

100W次插入，用时191.684s，平均每秒插入5216条数据，这个速度还是非常非常快的，所以放心用吧，
不用担心速度问题啦。

Stop / remove所有Docker容器笔记

一行代码停止或者删除全部的Docker容器

1
2
docker stop $(docker ps -a -q)
docker rm $(docker ps -a -q)

个人笔记

以docker stop为例，输入docker stop，显示

1
2
3
4
5
6
7
docker: "stop" requires a minimum of 1 argument.
See 'docker stop --help'.

Usage:	docker stop [OPTIONS] CONTAINER [CONTAINER...]

Stop a running container.
Sending SIGTERM and then SIGKILL after a grace period

帮助说明docker stop 可以接受一个以空格分隔的容器数组，只要把全部的容器名称以空格连接就可以了。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
> docker ps -a -q
Usage:	docker ps [OPTIONS]

List containers

  -a, --all=false       Show all containers (default shows just running)
  --before=             Show only container created before Id or Name
  -f, --filter=[]       Filter output based on conditions provided
  --format=             Pretty-print containers using a Go template
  --help=false          Print usage
  -l, --latest=false    Show the latest created container, include non-running
  -n=-1                 Show n last created containers, include non-running
  --no-trunc=false      Don't truncate output
  -q, --quiet=false     Only display numeric IDs
  -s, --size=false      Display total file sizes
  --since=              Show created since Id or Name, include non-running
  • -a表示全部的容器
  • -q表示只显示容器ID

这样就可以取得全部的以换行符分隔的容器ID列表，再通过$()操作将换行符替换，就可以了。

Thinking in API

当时全世界只有一种语言和一样的话。
当人们由东方迁移的时候，在史纳尔地方找到了一块平原，就在那里住下了。
他们彼此说：「来，我们做砖，用火烧透。」他们遂拿砖当石，拿沥青代灰泥。
然后彼此说：「来，让我们建造一城一塔，塔顶摩天，好给我们作记念，免得我们在全地面上分散了!」
上主遂下来，要看看世人所造的城和塔。
上主说：「看，他们都是一个民族，都说一样的语言。他们如今就开始做这事；以后他们所想做的，就没有不成功的了。
来，我们下去，混乱他们的语言，使他们彼此语言不通。」
于是上主将他们分散到全地面，他们遂停止建造那城。
为此人称那地为「巴贝耳，」因为上主在那里混乱了全地的语言，且从那里将他们分散到全地面。

话说上帝为了让人类不能做他们所想于是混乱了大家的语言，使大家语言不通。我们写程序的也被各种不同的
语言所隔离，各不相通，同一个库不同的语言都有各自的实现，有好有差。大家为什么就不能都说同一个语言呢？
PHP的库我写Python时也想用，这难道不行吗？

我的想法是将各种库转换为我们程序员通过的语言，简单明了，就是emoji表情一样😂，以下是我的想法，有时间实现一下，
一个以API为中心的数据平台：

  • 数据通过JSON为数据序列化
  • 使用基于RESTful的HTTP请求进行通信（可以使用swagger）
  • 统一的接口页面
  • 同风格的展示页面