guke
37fd51a498
背景 / 目标
客户端埋点从「采集」到「上报落库」全链路存在丢失(采集端丢帧、网络丢包、上报失败),现有 analytics_event(行为事件)无法度量这条链路的健康度。本 MR 引入一条与行为埋点完全独立的自报计数(selfstat)链路:客户端周期上报「自 epoch 起算的累计计数」,服务端只 append 存原始快照,admin 查询时在 Python 侧差分聚合出两段成功率(埋点成功率 / 上报成功率),支持总览 / 按天趋势 / 按维度下钻。
改动概览
① App 侧 — 上报接入(不强制登录)
新增 POST /api/v1/analytics/selfstat:一份快照 = 设备头 + N 条 event 累计计数,单事务落库,返回 snapshot_id。
稳态兜底:落库异常不裸奔 500,logger.exception 后回 503(计数链路要稳,不阻塞端上主流程)。
events 允许空列表(某次只报设备级计数也合法),上限 200 条/次;与行为埋点 min_length=1 有意不同。
② Admin 侧 — 健康度聚合(只读,需 admin 鉴权)
新增 GET /admin/api/analytics-health/{overview,trend,breakdown}。
差分聚合:原始累计快照按 (device_id, epoch_id, event) 分区、created_at 升序做相邻差分;分区首行增量=累计值,负值(epoch 重置 / 乱序)夹 0;每条增量按其快照 created_at 归入北京天桶。
基线行:取 date_from 左侧每分区最后一条快照,保证区间第一条增量正确(参与差分后丢弃)。
两段率(分母为 0 → null):
埋点成功率 track_success_rate = (attempted − drop_capture) / attempted
上报成功率 report_success_rate = delivered / (delivered + drop_undelivered)
breakdown 维度限 event | app_ver | oem(路由正则校验);结果按上报成功率升序(最差在前,None 垫底)。
③ 数据模型 / 基建
新表 analytics_selfstat(快照头 + 设备维度 app_ver/oem/os + 设备级诊断量 batches_attempted/ok/fail、retries、queue_depth、端 sent_at + 服务端权威 created_at)与 analytics_selfstat_event(四类累计计数,FK → 快照头)。
Alembic 迁移 11c44afbea58(down_revision admin_user_plain_password):建两表 + 索引。
模型登记进 app/models/__init__.py(供 Alembic 发现);路由挂进 app/admin/main.py。
关键设计取舍
只存原始累计、查询时 Python 差分:admin 低频、量级小,跨 PG/SQLite 无方言坑(与 cps.py/coupon_data.py 同款约定)。
选「最新一条」用 max(id) 而非 max(created_at):id 严格单调,规避 SQLite 秒级时间戳撞车的歧义。
tz 口径统一:SQLite 返回 naive UTC、PG 返回 aware UTC,过滤前统一转 naive UTC 再比较。
基线子查询无下界:扫 date_from 左侧全量(spec §7 已接受的取舍;量级变大再上物化 rollup)。
测试(+14,全绿)
tests/test_analytics_selfstat.py(3):正常落库 / 空 events / 落库异常回 503。
tests/test_analytics_health.py(11):差分逻辑(首行=累计值、相邻差、epoch 重置换分区、乱序夹 0、同时间戳按 id 定序)、两段率公式、零分母→None、北京天边界、端点(overview 鉴权 / overview / breakdown)。
迁移 / 部署注意
部署需执行 alembic upgrade head(建两张新表)。纯新增表,无回填、无破坏性改动,向后兼容。
客户端需按约定 payload(snake_case、累计语义)对接 /api/v1/analytics/selfstat;admin 前端(独立仓 shaguabijia-admin-web)消费三个 analytics-health 端点,不在本 MR 范围。
---------
Co-authored-by: guke <guke@autohome.com.cn>
Reviewed-on: #127
113 lines
4.3 KiB
Python
113 lines
4.3 KiB
Python
"""Admin 后台 FastAPI app(独立进程)。
|
|
|
|
启动:uvicorn app.admin.main:admin_app --host 127.0.0.1 --port 8771
|
|
复用 App 的 DB/models/repositories/integrations;鉴权独立(admin JWT,见 app/admin/security.py)。
|
|
现有 app.main:app 不 import 本模块,两进程互不影响。
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import logging
|
|
from collections.abc import AsyncIterator
|
|
from contextlib import asynccontextmanager
|
|
|
|
from fastapi import FastAPI
|
|
from fastapi.middleware.cors import CORSMiddleware
|
|
|
|
from app.admin.routers.ad_audit import router as ad_audit_router
|
|
from app.admin.routers.ad_config import router as ad_config_router
|
|
from app.admin.routers.ad_revenue import router as ad_revenue_router
|
|
from app.admin.routers.admins import router as admins_router
|
|
from app.admin.routers.audit import router as audit_router
|
|
from app.admin.routers.auth import router as auth_router
|
|
from app.admin.routers.comparison import router as comparison_router
|
|
from app.admin.routers.config import router as config_router
|
|
from app.admin.routers.coupon_data import router as coupon_data_router
|
|
from app.admin.routers.cps import router as cps_router
|
|
from app.admin.routers.dashboard import router as dashboard_router
|
|
from app.admin.routers.device_liveness import router as device_liveness_router
|
|
from app.admin.routers.ops_stat_config import router as ops_stat_config_router
|
|
from app.admin.routers.analytics_health import router as analytics_health_router
|
|
from app.admin.routers.event_logs import router as event_logs_router
|
|
from app.admin.routers.feedback import router as feedback_router
|
|
from app.admin.routers.feedback_qr import router as feedback_qr_router
|
|
from app.admin.routers.onboarding import router as onboarding_router
|
|
from app.admin.routers.ops_marquee_seed import router as ops_marquee_seed_router
|
|
from app.admin.routers.price_report import router as price_report_router
|
|
from app.admin.routers.roles import router as roles_router
|
|
from app.admin.routers.users import router as users_router
|
|
from app.admin.routers.wallet import router as wallet_router
|
|
from app.admin.routers.withdraw import router as withdraw_router
|
|
from app.core.config import settings
|
|
from app.core.logging import setup_logging
|
|
|
|
setup_logging(debug=settings.APP_DEBUG)
|
|
logger = logging.getLogger("shagua.admin")
|
|
|
|
|
|
@asynccontextmanager
|
|
async def lifespan(_: FastAPI) -> AsyncIterator[None]:
|
|
logger.info(
|
|
"admin app started env=%s db=%s",
|
|
settings.APP_ENV,
|
|
settings.DATABASE_URL.split("://", 1)[0],
|
|
)
|
|
yield
|
|
logger.info("admin app shutting down")
|
|
|
|
|
|
admin_app = FastAPI(
|
|
title=f"{settings.APP_NAME} · Admin",
|
|
version="0.1.0",
|
|
docs_url="/admin/docs" if not settings.is_prod else None,
|
|
redoc_url=None,
|
|
lifespan=lifespan,
|
|
)
|
|
|
|
# admin 前端独立部署。生产同域(nginx)无需 CORS;本地 next dev 跨域需放行开发源。
|
|
_dev_origins = [
|
|
"http://localhost:3001",
|
|
"http://127.0.0.1:3001",
|
|
"http://localhost:3000",
|
|
"http://127.0.0.1:3000",
|
|
]
|
|
_origins = settings.cors_origins_list or ([] if settings.is_prod else _dev_origins)
|
|
if _origins:
|
|
admin_app.add_middleware(
|
|
CORSMiddleware,
|
|
allow_origins=_origins,
|
|
allow_credentials=True,
|
|
allow_methods=["*"],
|
|
allow_headers=["*"],
|
|
)
|
|
|
|
|
|
@admin_app.get("/admin/api/health", tags=["meta"])
|
|
def health() -> dict[str, str]:
|
|
return {"status": "ok", "service": "admin"}
|
|
|
|
|
|
admin_app.include_router(auth_router)
|
|
admin_app.include_router(dashboard_router)
|
|
admin_app.include_router(device_liveness_router)
|
|
admin_app.include_router(ops_stat_config_router)
|
|
admin_app.include_router(ops_marquee_seed_router)
|
|
admin_app.include_router(users_router)
|
|
admin_app.include_router(onboarding_router)
|
|
admin_app.include_router(wallet_router)
|
|
admin_app.include_router(withdraw_router)
|
|
admin_app.include_router(price_report_router)
|
|
admin_app.include_router(feedback_router)
|
|
admin_app.include_router(event_logs_router)
|
|
admin_app.include_router(analytics_health_router)
|
|
admin_app.include_router(feedback_qr_router)
|
|
admin_app.include_router(admins_router)
|
|
admin_app.include_router(roles_router)
|
|
admin_app.include_router(audit_router)
|
|
admin_app.include_router(config_router)
|
|
admin_app.include_router(comparison_router)
|
|
admin_app.include_router(cps_router)
|
|
admin_app.include_router(coupon_data_router)
|
|
admin_app.include_router(ad_audit_router)
|
|
admin_app.include_router(ad_config_router)
|
|
admin_app.include_router(ad_revenue_router)
|