# batch-mct-demo

**Repository Path**: c-java-father/batch-mct-demo

## Basic Information

- **Project Name**: batch-mct-demo
- **Description**: batch + mysql + clickhouse + tdengine

- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2026-01-09
- **Last Updated**: 2026-01-16

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# batch + mysql + clickhouse + tdengine

2亿条数据-20G内存-8核cpu
Intel(R) Xeon(R) CPU           E5603  @ 1.60GHz


|   数据库   | 插入速度 | 9000万之后查询 | 占用硬盘 | CPU占用 |
| :--------: | :------: | :------------: | :------: | :-----: |
|  tdengine  | 40万/秒 |   60秒-180秒   |    3G    |   40%   |
| clickhouse | 60万/秒 |      90秒      |   23G   |  380%  |

# TDEngine Cloud example

## Introduction

Use Spring Boot to build a simple web application, and deploy it to connect to TDEngine Cloud.

## Usage

1. build the project

```
mvn clean package -Dmaven.test.skip=true
```

2. replace the current url with the database configuration in cloud page `Programming -> Java -> Config`, and run the project

```
java -jar target/example-1.0-SNAPSHOT.jar --spring.datasource.url="jdbc:TAOS-RS://xxxxx"
```

### Check the result

3. open the browser and visit the following url

```
http://localhost:8888/meter/list
```

5. create sub-table `mct-demo` of meters, sub_table name is only allowed to contain letters, numbers, and underscores.

```aidl
http://localhost:8888/meter/mct-demo/create
```

6. insert data to sub-table `mct-demo` of meters

```aidl
http://localhost:8888/meter/mct-demo/insert
```

7. query last row from sub-table `mct-demo` of meters

```aidl
http://localhost:8888/meter/mct-demo/last_row


-- 1. 创建数据库（如果不存在）
CREATE DATABASE IF NOT EXISTS iot;
USE iot;

-- 2. 创建超级表（STable）
-- 创建带性能优化参数的超级表
CREATE STABLE IF NOT EXISTS meters (
    ts TIMESTAMP,
    current FLOAT,
    voltage INT,
    phase FLOAT
) TAGS (
    group_id INT,
    location NCHAR(100)
) 
KEEP 3650,                      -- 数据保留3650天（10年）
DAYS 10,                        -- 每个数据文件存储10天的数据
BLOCKS 100,                     -- 每个VNODE的缓存块数
QUORUM 2,                       -- 写入确认数
WAL 1;                          -- 启用WAL

-- 3. 使用超级表创建子表
CREATE TABLE IF NOT EXISTS test USING meters TAGS ('Los Angeles', 1);
```

8. batch insert data to tdengine

```aidl
http://localhost:8888/weather/batch/threads/100000000/5000/false/taos

taos/clickhouse/mysql

{
  "totalRecords": 90000000,
  "successfulRecords": 90000000,
  "failedRecords": 0,
  "executionTime": 1751935,
  "recordsPerSec": 51371.76892978336,
  "message": "超级表优化批量插入完成",
  "success": true
}

http://localhost:8888/weather/list/10/90000000
```

9. batch test deepseek生成代码报错

```aidl
# 1. 生成并插入10万条北京天气数据到所有数据库
curl -X GET "http://localhost:8888/api/weather/generate-and-insert?count=10000&target=all"

# 2. 多城市批量插入
curl -X GET "http://localhost:8888/api/weather/multi-city-insert?count=500000&target=all"

# 3. 性能测试（100万条数据）
curl -X GET "http://localhost:8888/api/weather/performance-test?count=1000000&batchSize=10000"

# 4. 查看数据库统计
curl "http://localhost:8888/api/weather/stats"

# 5. 查看线程池状态
curl "http://localhost:8888/api/weather/thread-pool-status"
```