添加prefect调度

.env.example

@@ -0,0 +1,19 @@
+# Feishu
+FEISHU_APP_ID=cli_xxx
+FEISHU_APP_SECRET=your_feishu_app_secret
+FEISHU_BASE_URL=https://your-domain.feishu.cn/base/app_token?table=tbl_xxx&view=vew_xxx
+FEISHU_CHAT_ID=oc_xxx
+
+# Prefect
+PREFECT_API_URL=http://host.docker.internal:4200/api
+PREFECT_UI_PORT=4200
+
+# Database - Analytics
+ANALYTICS_DB_HOST=127.0.0.1
+ANALYTICS_DB_PORT=5432
+ANALYTICS_DB_USER=postgres
+ANALYTICS_DB_PASSWORD=your_db_password
+ANALYTICS_DB_NAME=ids_etl
+
+# Optional
+BATCH_SIZE=500

.gitignore

@@ -3,3 +3,6 @@
 __pycache__/
 .DS_Store
 .vscode/
+project/
+project.zip
+example

DOCKER_PREFECT_USAGE.md

@@ -0,0 +1,269 @@
+# Docker + Prefect 使用说明
+
+当前方案是：
+
+- 使用 [prefect.yaml](/Users/idss/YXT_Project/logistics-service/prefect.yaml) 管理 deployment
+- 使用 Docker 容器运行 Prefect worker
+- 由 Prefect Server 统一调度
+
+当前只有一条调度：
+
+- `12:00` 执行物流未发货数据同步，并写入飞书多维表格
+
+## 1. 创建 Docker 镜像
+
+在项目目录执行：
+
+```bash
+cd /Users/idss/YXT_Project/logistics-service
+docker build -t logistics-service-prefect:0.1.0 .
+```
+
+镜像构建特点：
+
+- 基础镜像：`python:3.12-slim-bookworm`
+- 使用 `uv` 安装依赖
+- 使用清华 `apt` 和 `PyPI` 源
+- 依赖来源：[pyproject.toml](/Users/idss/YXT_Project/logistics-service/pyproject.toml)
+
+## 2. 启动 Docker 容器
+
+推荐使用 Compose。
+
+项目里的 [docker-compose.yaml](/Users/idss/YXT_Project/logistics-service/docker-compose.yaml) 已经配置了：
+
+- `.env`
+- `PREFECT_API_URL`
+- `FEISHU_CHAT_ID`
+- `host.docker.internal`
+- `uv_cache` 持久化卷
+- CPU / 内存资源限制
+
+启动：
+
+```bash
+cd /Users/idss/YXT_Project/logistics-service
+docker compose up -d --build
+```
+
+停止：
+
+```bash
+docker compose down
+```
+
+查看运行状态：
+
+```bash
+docker ps
+```
+
+当前容器名是：
+
+```bash
+prefect-logistics-worker
+```
+
+## 3. 容器启动后会做什么
+
+容器启动后：
+
+1. 加载 `.env`
+2. 创建 `/app/logs`
+3. 启动 `supervisord`
+4. 再由 [supervisord.conf](/Users/idss/YXT_Project/logistics-service/supervisord.conf) 启动：
+
+```bash
+prefect worker start --pool "logistics-service-pool" --type process
+```
+
+也就是说，这个容器的职责不是直接跑 flow，而是作为 Prefect worker 等待调度。
+
+## 4. 注册 Prefect 调度
+
+调度定义在 [prefect.yaml](/Users/idss/YXT_Project/logistics-service/prefect.yaml)。
+
+当前 work pool 是：
+
+```bash
+logistics-service-pool
+```
+
+注册 deployment 的方式：
+
+### 方式 A：在本机或服务器上直接执行
+
+前提是当前环境里有 `prefect` CLI，并且能访问 `PREFECT_API_URL`：
+
+```bash
+cd /Users/idss/YXT_Project/logistics-service
+prefect deploy --all
+```
+
+### 方式 B：在容器里执行
+
+如果宿主机没有 `prefect` 命令，可以直接在容器里执行：
+
+```bash
+docker exec -it prefect-logistics-worker sh -lc 'cd /app && prefect deploy --all'
+```
+
+注意：
+
+- 这条命令要求容器里的 `PREFECT_API_URL` 可访问
+- 如果你看到 `ConnectError`，通常是 `.env` 里的 `PREFECT_API_URL` 地址在容器里不可达
+
+## 5. 自动执行是怎么发生的
+
+完整流程如下：
+
+1. `prefect deploy --all` 把 [prefect.yaml](/Users/idss/YXT_Project/logistics-service/prefect.yaml) 注册到 Prefect Server
+2. Prefect Server 根据 cron 在每天 `12:00` 触发：
+   - `logistics-bitable-sync-daily`
+3. `prefect-logistics-worker` 容器里的 worker 从 `logistics-service-pool` 拉取任务
+4. worker 执行 [prefect_logistics_scheduler.py](/Users/idss/YXT_Project/logistics-service/prefect_logistics_scheduler.py) 里的 flow
+5. flow 调用 [import_logistics_to_feishu_bitable.py](/Users/idss/YXT_Project/logistics-service/import_logistics_to_feishu_bitable.py)：
+   - 查询 PostgreSQL
+   - 删除多维表旧记录
+   - 写入新记录
+6. 同步完成或失败后，使用 `FEISHU_APP_ID`、`FEISHU_APP_SECRET` 和 `FEISHU_CHAT_ID` 推送结果到飞书群
+
+## 6. 单次手动执行
+
+手动跑一次同步：
+
+```bash
+python /Users/idss/YXT_Project/logistics-service/prefect_logistics_scheduler.py --mode sync
+```
+
+如果要在 Docker 里手动执行：
+
+```bash
+docker run --rm \
+  --env-file /Users/idss/YXT_Project/logistics-service/.env \
+  -e PREFECT_API_URL=http://host.docker.internal:4200/api \
+  --add-host host.docker.internal:host-gateway \
+  logistics-service-prefect:0.1.0 \
+  python prefect_logistics_scheduler.py --mode sync
+```
+
+## 7. Docker 中的日志
+
+现在日志分成三层：
+
+### 7.1 容器标准输出日志
+
+查看 worker 容器日志：
+
+```bash
+docker logs -f prefect-logistics-worker
+```
+
+或者：
+
+```bash
+cd /Users/idss/YXT_Project/logistics-service
+docker compose logs -f
+```
+
+这里通常能看到：
+
+- worker 启动日志
+- Prefect 调度执行日志
+- Python 进程报错堆栈
+
+### 7.2 容器内本地文件日志
+
+worker 的 supervisor 日志会写到：
+
+- `/app/logs/prefect_worker.out.log`
+- `/app/logs/prefect_worker.err.log`
+
+查看方式：
+
+```bash
+docker exec -it prefect-logistics-worker sh
+tail -f /app/logs/prefect_worker.out.log
+```
+
+### 7.3 Prefect UI 日志
+
+在 Prefect UI 的 `Logs` 面板里也能看到 flow / task 日志。
+
+如果 UI 没看到新日志，通常说明：
+
+- worker 还没重启到最新代码
+- 或本次 run 还是旧 deployment / 旧镜像执行的
+
+## 8. 常见问题
+
+### 8.1 `prefect: command not found`
+
+说明宿主机没有安装 Prefect CLI。  
+可改为在容器里执行：
+
+```bash
+docker exec -it prefect-logistics-worker sh -lc 'cd /app && prefect deploy --all'
+```
+
+### 8.2 `httpx.ConnectError: All connection attempts failed`
+
+说明容器里访问不到 `PREFECT_API_URL`。  
+常见原因：
+
+- `.env` 里的 `PREFECT_API_URL` 写成了 `localhost`
+- 容器里访问不到宿主机 Prefect API
+
+当前 compose 已经加了：
+
+```yaml
+extra_hosts:
+  - "host.docker.internal:host-gateway"
+```
+
+所以更推荐把 `.env` 里的 `PREFECT_API_URL` 配成类似：
+
+```bash
+PREFECT_API_URL=http://host.docker.internal:4200/api
+```
+
+### 8.3 飞书返回 `91403 Forbidden`
+
+说明当前应用对目标多维表没有足够权限。  
+需要检查：
+
+- 应用是否已添加到目标 Base
+- 是否具备可编辑权限
+- 是否允许新增记录 / 删除记录 / 新增字段
+
+### 8.4 没有收到飞书群通知
+
+需要检查：
+
+- `.env` 中是否配置了 `FEISHU_CHAT_ID`
+- 当前应用是否具备发送消息权限
+- 该应用对应机器人是否已加入目标群
+
+## 9. 最推荐的执行方式
+
+如果你要的是“服务器上长期稳定自动执行”，建议按这个顺序：
+
+1. 构建镜像
+2. 准备 `.env`
+3. 启动 worker 容器
+
+```bash
+cd /Users/idss/YXT_Project/logistics-service
+docker compose up -d --build
+```
+
+4. 在能连到 `PREFECT_API_URL` 的环境里注册 deployment
+
+```bash
+docker exec -it prefect-logistics-worker sh -lc 'cd /app && prefect deploy --all'
+```
+
+5. 到 Prefect UI 检查：
+   - deployment 是否已注册
+   - work pool 是否是 `logistics-service-pool`
+   - schedule 是否是每天 `12:00`

Dockerfile

@@ -0,0 +1,48 @@
+FROM ghcr.io/astral-sh/uv:latest AS uv
+
+FROM python:3.12-slim-bookworm
+
+COPY --from=uv /uv /bin/uv
+
+ARG UV_INDEX_URL=https://pypi.tuna.tsinghua.edu.cn/simple
+
+ENV PYTHONUNBUFFERED=1 \
+    PYTHONDONTWRITEBYTECODE=1 \
+    UV_SYSTEM_PYTHON=1 \
+    UV_INDEX_URL=${UV_INDEX_URL} \
+    TZ=Asia/Shanghai \
+    PYTHONPATH="/app:${PYTHONPATH}"
+
+RUN echo "# 清华大学 Debian 镜像源配置\n\
+deb https://mirrors.tuna.tsinghua.edu.cn/debian/ bookworm main contrib non-free non-free-firmware\n\
+deb-src https://mirrors.tuna.tsinghua.edu.cn/debian/ bookworm main contrib non-free non-free-firmware\n\
+\n\
+deb https://mirrors.tuna.tsinghua.edu.cn/debian-security/ bookworm-security main contrib non-free non-free-firmware\n\
+deb-src https://mirrors.tuna.tsinghua.edu.cn/debian-security/ bookworm-security main contrib non-free non-free-firmware\n\
+\n\
+deb https://mirrors.tuna.tsinghua.edu.cn/debian/ bookworm-updates main contrib non-free non-free-firmware\n\
+deb-src https://mirrors.tuna.tsinghua.edu.cn/debian/ bookworm-updates main contrib non-free non-free-firmware" > /etc/apt/sources.list
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    supervisor \
+    tzdata \
+    && ln -snf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime \
+    && echo "Asia/Shanghai" > /etc/timezone \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /app
+
+COPY pyproject.toml /app/pyproject.toml
+
+RUN uv pip install -r /app/pyproject.toml
+
+COPY . /app
+
+RUN chmod +x /app/docker-entrypoint.sh \
+    && mkdir -p /app/logs \
+    && mkdir -p /var/log/supervisor
+
+COPY supervisord.conf /etc/supervisor/conf.d/supervisord.conf
+
+ENTRYPOINT ["/bin/sh", "/app/docker-entrypoint.sh"]
+CMD ["/usr/bin/supervisord", "-c", "/etc/supervisor/conf.d/supervisord.conf"]

docker-compose.yaml

@@ -0,0 +1,32 @@
+services:
+  prefect-logistics-worker:
+    build:
+      context: .
+      dockerfile: Dockerfile
+      args:
+        - UV_INDEX_URL=https://pypi.tuna.tsinghua.edu.cn/simple
+    image: logistics-service-prefect:0.1.0
+    container_name: prefect-logistics-worker
+    restart: unless-stopped
+    env_file:
+      - .env
+    environment:
+      - PYTHONUNBUFFERED=1
+      - PREFECT_API_URL=${PREFECT_API_URL:-http://host.docker.internal:4200/api}
+    extra_hosts:
+      - "host.docker.internal:host-gateway"
+    volumes:
+      - .:/app
+      - ./logs:/app/logs
+      - uv_cache:/root/.cache/uv
+    deploy:
+      resources:
+        limits:
+          cpus: "2.0"
+          memory: 4G
+        reservations:
+          cpus: "0.5"
+          memory: 512M
+
+volumes:
+  uv_cache:

docker-entrypoint.sh

@@ -0,0 +1,14 @@
+#!/bin/sh
+set -eu
+
+APP_DIR="/app"
+
+if [ -f "$APP_DIR/.env" ]; then
+  set -a
+  . "$APP_DIR/.env"
+  set +a
+fi
+
+mkdir -p "$APP_DIR/logs"
+
+exec "$@"

import_logistics_to_feishu_bitable.py

@@ -1,8 +1,6 @@
 #!/usr/bin/env python3
-import csv
 import json
 import os
-import sys
 import time
 from datetime import datetime
 from pathlib import Path
@@ -23,9 +21,9 @@ except ImportError:
 
 
 ENV_PATH = Path(".env")
-DEFAULT_CSV_PATH = Path("物流.csv")
 DEFAULT_BATCH_SIZE = 500
-DEFAULT_SOURCE = "db"
+DEFAULT_HTTP_RETRIES = 3
+DEFAULT_HTTP_RETRY_DELAY = 2.0
 
 DB_QUERY = """
 SELECT
@@ -101,6 +99,11 @@ FIELD_TYPE_HINTS = {
 }
 
 
+def log(message: str, *, level: str = "INFO") -> None:
+    timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
+    print(f"[{timestamp}] [{level}] {message}")
+
+
 def load_env(path: Path) -> None:
     if not path.exists():
         return
@@ -154,6 +157,8 @@ def feishu_request(
     *,
     token: Optional[str] = None,
     payload: Optional[dict] = None,
+    retries: int = DEFAULT_HTTP_RETRIES,
+    retry_delay: float = DEFAULT_HTTP_RETRY_DELAY,
 ) -> dict:
     headers = {"Content-Type": "application/json; charset=utf-8"}
     if token:
@@ -163,18 +168,53 @@ def feishu_request(
     if payload is not None:
         data = json.dumps(payload, ensure_ascii=False).encode("utf-8")
 
+    for attempt in range(1, retries + 1):
         req = request.Request(url=url, method=method, headers=headers, data=data)
         try:
             with request.urlopen(req, timeout=60) as resp:
                 body = resp.read().decode("utf-8")
+            break
         except error.HTTPError as exc:
             detail = exc.read().decode("utf-8", errors="replace")
+            should_retry = exc.code in {429, 500, 502, 503, 504}
+            log(
+                f"Feishu API HTTP 异常: method={method} status={exc.code} "
+                f"url={url} attempt={attempt}/{retries} detail={detail[:500]}",
+                level="ERROR",
+            )
+            if should_retry and attempt < retries:
+                sleep_seconds = retry_delay * attempt
+                log(
+                    f"Feishu API 将重试: method={method} status={exc.code} "
+                    f"next_retry_in={sleep_seconds:.1f}s",
+                    level="WARN",
+                )
+                time.sleep(sleep_seconds)
+                continue
             raise RuntimeError(f"Feishu API 请求失败: {exc.code} {detail}") from exc
         except error.URLError as exc:
+            log(
+                f"Feishu API 网络异常: method={method} url={url} "
+                f"attempt={attempt}/{retries} error={exc}",
+                level="ERROR",
+            )
+            if attempt < retries:
+                sleep_seconds = retry_delay * attempt
+                log(
+                    f"Feishu API 网络异常后重试: method={method} next_retry_in={sleep_seconds:.1f}s",
+                    level="WARN",
+                )
+                time.sleep(sleep_seconds)
+                continue
             raise RuntimeError(f"无法连接飞书 API: {exc}") from exc
 
     result = json.loads(body)
     if result.get("code") != 0:
+        log(
+            f"Feishu API 业务异常: method={method} url={url} "
+            f"code={result.get('code')} msg={result.get('msg')}",
+            level="ERROR",
+        )
         raise RuntimeError(
             f"Feishu API 返回错误: code={result.get('code')} msg={result.get('msg')} body={body}"
         )
@@ -196,6 +236,44 @@ def get_tenant_access_token(app_id: str, app_secret: str) -> str:
     return result["tenant_access_token"]
 
 
+def send_feishu_group_message(text: str) -> None:
+    app_id = os.getenv("FEISHU_APP_ID")
+    app_secret = os.getenv("FEISHU_APP_SECRET")
+    chat_id = os.getenv("FEISHU_CHAT_ID")
+    if not app_id or not app_secret or not chat_id:
+        raise RuntimeError("缺少 FEISHU_APP_ID / FEISHU_APP_SECRET / FEISHU_CHAT_ID")
+
+    token = get_tenant_access_token(app_id, app_secret)
+    url = "https://open.feishu.cn/open-apis/im/v1/messages?receive_id_type=chat_id"
+    payload = {
+        "receive_id": chat_id,
+        "msg_type": "text",
+        "content": json.dumps({"text": text}, ensure_ascii=False),
+    }
+    feishu_request("POST", url, token=token, payload=payload)
+
+
+def build_success_message(summary: Dict[str, int]) -> str:
+    base_url = os.getenv("FEISHU_BASE_URL", "")
+    started_at = summary.get("started_at", "")
+    finished_at = summary.get("finished_at", "")
+    message = (
+        "截止到中午12点，超48小时未发货的订单数据同步完成\n"
+        f"新增记录数: {summary['created']}\n"
+    )
+    if base_url:
+        message += f"\n多维表链接: {base_url}"
+    return message
+
+
+def build_failure_message(error_text: str, finished_at: str) -> str:
+    return (
+        "物流多维表同步失败\n"
+        f"执行时间: {finished_at}\n"
+        f"错误信息: {error_text}"
+    )
+
+
 def paginate_fields(app_token: str, table_id: str, token: str) -> List[dict]:
     fields: List[dict] = []
     page_token: Optional[str] = None
@@ -210,14 +288,45 @@ def paginate_fields(app_token: str, table_id: str, token: str) -> List[dict]:
         )
         result = feishu_request("GET", url, token=token)
         data = get_response_data(result)
-        fields.extend(data.get("items", []))
+        items = data.get("items", [])
+        fields.extend(items)
         if not data.get("has_more"):
             break
         page_token = data.get("page_token")
 
+    log(f"字段读取完成: total={len(fields)}")
     return fields
 
 
+def list_all_record_ids(app_token: str, table_id: str, token: str) -> List[str]:
+    record_ids: List[str] = []
+    page_token: Optional[str] = None
+
+    while True:
+        query = {"page_size": 100}
+        if page_token:
+            query["page_token"] = page_token
+        url = (
+            f"https://open.feishu.cn/open-apis/bitable/v1/apps/{app_token}"
+            f"/tables/{table_id}/records?{parse.urlencode(query)}"
+        )
+        result = feishu_request("GET", url, token=token)
+        data = get_response_data(result)
+        items = data.get("items", [])
+        for item in items:
+            record_id = item.get("record_id") or item.get("id")
+            if record_id:
+                record_ids.append(record_id)
+        if not data.get("has_more"):
+            break
+        page_token = data.get("page_token")
+        if not page_token:
+            break
+
+    log(f"目标表现有记录统计: total={len(record_ids)}")
+    return record_ids
+
+
 def create_field(app_token: str, table_id: str, token: str, field_name: str, field_type: int) -> dict:
     url = f"https://open.feishu.cn/open-apis/bitable/v1/apps/{app_token}/tables/{table_id}/fields"
     payload = {
@@ -232,15 +341,6 @@ def create_field(app_token: str, table_id: str, token: str, field_name: str, fie
     return field
 
 
-def normalize_value(raw: Optional[str]) -> Optional[str]:
-    if raw is None:
-        return None
-    value = raw.strip()
-    if not value or value.upper() == "NULL":
-        return None
-    return value
-
-
 def convert_for_field(value: Optional[str], field_type: int):
     if value is None:
         return None
@@ -264,31 +364,24 @@ def convert_for_field(value: Optional[str], field_type: int):
         return value.lower() in {"1", "true", "yes", "y", "是"}
 
     if field_type == FIELD_TYPE_DATETIME:
-        # CSV 中是 `2026-03-15 14:20:30+08` 这种格式，转换成毫秒时间戳。
+        for pattern in ("%Y-%m-%d", "%Y-%m-%d %H:%M:%S+08", "%Y-%m-%d %H:%M:%S"):
             try:
-            dt = datetime.strptime(value, "%Y-%m-%d %H:%M:%S+08")
+                dt = datetime.strptime(value, pattern)
+                if pattern == "%Y-%m-%d":
+                    dt = datetime(dt.year, dt.month, dt.day)
                 return int(dt.timestamp() * 1000)
             except ValueError:
+                continue
         return value
 
     return value
 
 
-def read_csv_rows(csv_path: Path) -> Tuple[List[str], List[Dict[str, Optional[str]]]]:
-    with csv_path.open("r", newline="", encoding="utf-8-sig") as handle:
-        reader = csv.DictReader(handle)
-        headers = reader.fieldnames or []
-        rows: List[Dict[str, Optional[str]]] = []
-        for row in reader:
-            rows.append({key: normalize_value(value) for key, value in row.items()})
-    return headers, rows
-
-
 def normalize_db_value(value) -> Optional[str]:
     if value is None:
         return None
     if isinstance(value, datetime):
-        return value.strftime("%Y-%m-%d %H:%M:%S+08")
+        return value.strftime("%Y-%m-%d")
     return str(value).strip() or None
 
 
@@ -342,10 +435,10 @@ def ensure_fields(
                     FIELD_TYPE_HINTS.get(header, FIELD_TYPE_TEXT),
                 )
                 field_map[header] = created
-                print(f"已创建字段: {header}")
+                log(f"字段已创建: field_name={header}")
             except RuntimeError as exc:
                 missing_headers.append(header)
-                print(f"跳过缺失字段: {header} ({exc})")
+                log(f"字段创建失败，已跳过: field_name={header} error={exc}", level="WARN")
 
     return field_map, missing_headers
 
@@ -390,70 +483,127 @@ def batch_create_records(
     total_batches = (len(records) + batch_size - 1) // batch_size
 
     for batch_index, batch in enumerate(chunked(records, batch_size), start=1):
+        log(f"开始写入批次: batch={batch_index}/{total_batches} size={len(batch)}")
         feishu_request("POST", url, token=token, payload={"records": batch})
         created += len(batch)
-        print(f"已写入批次 {batch_index}/{total_batches}，累计 {created} 条")
+        log(f"写入批次完成: batch={batch_index}/{total_batches} created_total={created}")
         time.sleep(0.15)
 
     return created
 
 
+def batch_delete_records(
+    app_token: str,
+    table_id: str,
+    token: str,
+    record_ids: List[str],
+    batch_size: int,
+) -> int:
+    if not record_ids:
+        log("目标表当前没有历史记录，无需删除")
+        return 0
+
+    url = f"https://open.feishu.cn/open-apis/bitable/v1/apps/{app_token}/tables/{table_id}/records/batch_delete"
+    deleted = 0
+    total_batches = (len(record_ids) + batch_size - 1) // batch_size
+
+    for batch_index, batch in enumerate(chunked(record_ids, batch_size), start=1):
+        log(f"开始删除批次: batch={batch_index}/{total_batches} size={len(batch)}")
+        result = feishu_request("POST", url, token=token, payload={"records": batch})
+        data = get_response_data(result)
+        response_records = data.get("records", [])
+        if response_records:
+            failed = [item.get("record_id") for item in response_records if not item.get("deleted", False)]
+            if failed:
+                log(f"删除批次存在失败记录: failed_count={len(failed)}", level="ERROR")
+                raise RuntimeError(f"删除记录失败，record_ids={failed}")
+        deleted += len(batch)
+        log(f"删除批次完成: batch={batch_index}/{total_batches} deleted_total={deleted}")
+        time.sleep(0.15)
+
+    return deleted
+
+
 def print_permission_help(message: str) -> None:
-    print("飞书返回权限不足，当前应用还不能向这个多维表新增字段或记录。", file=sys.stderr)
+    log("飞书返回权限不足，当前应用还不能向这个多维表新增字段或记录。", level="ERROR")
-    print(f"原始错误: {message}", file=sys.stderr)
+    log(f"原始错误: {message}", level="ERROR")
-    print(
+    log(
         "请在飞书开放平台或多维表权限设置中，确认这个应用已被添加到该 Base，且具备多维表读取/写入权限。",
-        file=sys.stderr,
+        level="ERROR",
     )
-    print(
+    log(
         "如果你手动补齐缺失字段并授权写入，重新执行 `python3 import_logistics_to_feishu_bitable.py` 即可继续导入。",
-        file=sys.stderr,
+        level="ERROR",
     )
 
 
-def main() -> int:
+def execute_sync() -> Dict[str, int]:
     load_env(ENV_PATH)
 
     app_id = os.getenv("FEISHU_APP_ID")
     app_secret = os.getenv("FEISHU_APP_SECRET")
     base_url = os.getenv("FEISHU_BASE_URL")
-    source = os.getenv("DATA_SOURCE", DEFAULT_SOURCE).strip().lower()
-    csv_path = Path(os.getenv("CSV_PATH", str(DEFAULT_CSV_PATH)))
     batch_size = int(os.getenv("BATCH_SIZE", str(DEFAULT_BATCH_SIZE)))
 
     if not app_id or not app_secret or not base_url:
-        print("缺少 FEISHU_APP_ID / FEISHU_APP_SECRET / FEISHU_BASE_URL", file=sys.stderr)
+        raise RuntimeError("缺少 FEISHU_APP_ID / FEISHU_APP_SECRET / FEISHU_BASE_URL")
-        return 1
-    if source not in {"db", "csv"}:
-        print(f"不支持的数据源: {source}，仅支持 db / csv", file=sys.stderr)
-        return 1
-    if source == "csv" and not csv_path.exists():
-        print(f"CSV 文件不存在: {csv_path}", file=sys.stderr)
-        return 1
 
+    started_at = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
     app_token, table_id, _view_id = parse_base_url(base_url)
-    if source == "db":
+    log(f"导入任务开始: source=db table_id={table_id}")
     headers, rows = query_db_rows(DB_QUERY)
-        print(f"读取数据库完成，共 {len(rows)} 行")
+    log(f"读取数据库完成: rows={len(rows)}")
-    else:
-        headers, rows = read_csv_rows(csv_path)
-        print(f"读取 CSV 完成: {csv_path}，共 {len(rows)} 行")
 
     try:
         token = get_tenant_access_token(app_id, app_secret)
+        log("已获取 tenant_access_token")
         field_map, missing_headers = ensure_fields(app_token, table_id, token, headers)
+        log(f"字段检查完成: field_count={len(field_map)} missing_count={len(missing_headers)}")
+        existing_record_ids = list_all_record_ids(app_token, table_id, token)
+        deleted = batch_delete_records(app_token, table_id, token, existing_record_ids, batch_size)
         records = build_records(rows, field_map, missing_headers)
+        log(f"记录构建完成: source_rows={len(rows)} upload_records={len(records)}")
         created = batch_create_records(app_token, table_id, token, records, batch_size)
     except RuntimeError as exc:
         if "91403" in str(exc) or "Forbidden" in str(exc):
             print_permission_help(str(exc))
-            return 2
+            raise RuntimeError("飞书多维表权限不足") from exc
+        log(f"导入任务失败: error={exc}", level="ERROR")
         raise
 
     if missing_headers:
-        print(f"以下字段未写入，请在多维表中手动创建后重跑: {', '.join(missing_headers)}")
+        log(f"存在未写入字段: missing_count={len(missing_headers)}", level="WARN")
-    print(f"导入完成，共写入 {created} 条记录")
+    log(f"导入任务完成: deleted={deleted} created={created} source_rows={len(rows)}")
+    return {
+        "started_at": started_at,
+        "finished_at": datetime.now().strftime("%Y-%m-%d %H:%M:%S"),
+        "deleted": deleted,
+        "created": created,
+        "source_rows": len(rows),
+        "missing_count": len(missing_headers),
+    }
+
+
+def main() -> int:
+    try:
+        summary = execute_sync()
+        try:
+            send_feishu_group_message(build_success_message(summary))
+            log("飞书群通知发送成功")
+        except Exception as notify_exc:
+            log(f"飞书群通知发送失败: error={notify_exc}", level="ERROR")
         return 0
+    except RuntimeError as exc:
+        try:
+            send_feishu_group_message(
+                build_failure_message(str(exc), datetime.now().strftime("%Y-%m-%d %H:%M:%S"))
+            )
+            log("失败通知已发送到飞书群", level="WARN")
+        except Exception as notify_exc:
+            log(f"失败通知发送失败: error={notify_exc}", level="ERROR")
+        if str(exc) == "飞书多维表权限不足":
+            return 2
+        raise
 
 
 if __name__ == "__main__":

prefect.yaml

@@ -0,0 +1,24 @@
+name: logistics-service
+prefect-version: "3.0.0"
+
+build: []
+push: []
+pull: []
+
+deployments:
+  - name: logistics-bitable-sync-daily
+    version: "1.0.0"
+    description: "每天定时同步未发货物流数据到飞书多维表格。"
+    entrypoint: prefect_logistics_scheduler.py:logistics_bitable_sync_flow
+    parameters: {}
+    tags:
+      - logistics
+      - feishu
+      - bitable
+      - prefect
+    schedules:
+      - cron: "0 12 * * *"
+        timezone: "Asia/Shanghai"
+        active: true
+    work_pool:
+      name: logistics-service-pool

prefect_logistics_scheduler.py

@@ -0,0 +1,96 @@
+#!/usr/bin/env python3
+import argparse
+import json
+import os
+from datetime import datetime
+
+from prefect import flow, get_run_logger, task
+
+import import_logistics_to_feishu_bitable
+
+
+def send_feishu_group_message(text: str) -> None:
+    import_logistics_to_feishu_bitable.load_env(import_logistics_to_feishu_bitable.ENV_PATH)
+
+    app_id = os.getenv("FEISHU_APP_ID")
+    app_secret = os.getenv("FEISHU_APP_SECRET")
+    chat_id = os.getenv("FEISHU_CHAT_ID")
+    if not app_id or not app_secret or not chat_id:
+        raise RuntimeError("缺少 FEISHU_APP_ID / FEISHU_APP_SECRET / FEISHU_CHAT_ID")
+
+    token = import_logistics_to_feishu_bitable.get_tenant_access_token(app_id, app_secret)
+    url = "https://open.feishu.cn/open-apis/im/v1/messages?receive_id_type=chat_id"
+    payload = {
+        "receive_id": chat_id,
+        "msg_type": "text",
+        "content": json.dumps({"text": text}, ensure_ascii=False),
+    }
+    import_logistics_to_feishu_bitable.feishu_request("POST", url, token=token, payload=payload)
+
+
+@task(name="notify-feishu-chat", retries=3, retry_delay_seconds=[30, 60, 120], log_prints=True)
+def notify_feishu_chat_task(text: str) -> None:
+    logger = get_run_logger()
+    logger.info("开始推送飞书群通知")
+    send_feishu_group_message(text)
+    logger.info("飞书群通知推送完成")
+
+
+@task(name="run-logistics-sync", retries=2, retry_delay_seconds=[60, 180], log_prints=True)
+def run_logistics_sync_task() -> dict:
+    logger = get_run_logger()
+    logger.info("开始执行物流数据同步任务")
+    started_at = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
+
+    try:
+        summary = import_logistics_to_feishu_bitable.execute_sync()
+    except Exception as exc:
+        finished_at = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
+        logger.error("物流数据同步任务失败: %s", exc)
+        failure_text = (
+            "物流多维表同步失败\n"
+            f"执行时间: {finished_at}\n"
+            f"错误信息: {exc}"
+        )
+        send_feishu_group_message(failure_text)
+        raise
+
+    finished_at = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
+    logger.info("物流数据同步任务完成")
+    return {
+        "status": "success",
+        "started_at": started_at,
+        "finished_at": finished_at,
+        **summary,
+    }
+
+
+@flow(name="logistics-bitable-sync-flow", log_prints=True)
+def logistics_bitable_sync_flow() -> dict:
+    result = run_logistics_sync_task()
+    import_logistics_to_feishu_bitable.load_env(import_logistics_to_feishu_bitable.ENV_PATH)
+    base_url = os.getenv("FEISHU_BASE_URL", "")
+    success_text = (
+        "截止到中午12点，超48小时未发货的订单数据同步完成\n"
+        f"新增记录数: {result['created']}\n"
+    )
+    if base_url:
+        success_text += f"\n多维表链接: {base_url}"
+    notify_feishu_chat_task(success_text)
+    return result
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description="Prefect scheduler for logistics -> Feishu Bitable sync")
+    parser.add_argument("--mode", choices=["sync"], default="sync")
+    args = parser.parse_args()
+
+    if args.mode == "sync":
+        logistics_bitable_sync_flow()
+        return 0
+
+    return 1
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

pyproject.toml

@@ -0,0 +1,17 @@
+[project]
+name = "logistics-service-prefect-worker"
+version = "0.1.0"
+description = "Prefect worker for logistics to Feishu Bitable sync"
+readme = "README.md"
+requires-python = ">=3.12"
+dependencies = [
+    "prefect==3.6.8",
+    "psycopg>=3.3.1",
+    "psycopg-binary>=3.3.1",
+    "psycopg2-binary>=2.9.9",
+]
+
+[dependency-groups]
+dev = [
+    "pytest>=9.0.2",
+]

supervisord.conf

@@ -0,0 +1,10 @@
+[supervisord]
+nodaemon=true
+
+[program:logistics-service-worker]
+command=prefect worker start --pool "logistics-service-pool" --type process
+directory=/app
+autostart=true
+autorestart=true
+stderr_logfile=/app/logs/prefect_worker.err.log
+stdout_logfile=/app/logs/prefect_worker.out.log