性能:比价和领券分位数改为 PostgreSQL 聚合 (#161)

## 背景
- 比价记录页和领券记录页前端已经只拉当前页,并直接展示后端 summary。
- 原后端仍会将筛选区间内的全部耗时值取回 Python 计算分位数,生产数据量增大后会放大数据库读取和应用内存开销。

## 修改内容
- 比价记录:成功耗时 P5/P50/P95/P99、平均耗时以及中途退出耗时 P5/P50/P95 改用 PostgreSQL percentile_cont/AVG 聚合。
- 领券记录:发起数、完成数、平均耗时和完成耗时 P5/P50/P95/P99 合并为一条 PostgreSQL 聚合查询。
- 日期、用户、环境、状态、店铺和商品等筛选条件继续与列表共用,统计口径不变。
- SQLite 不支持 percentile_cont,仅在本地和测试环境回退读取耗时单列;不加载完整业务记录。
- API 字段与前端展示保持不变,无需前端改动。

## 验证
- 比价/领券及关联广告收益、点位、按券统计测试:26 passed。
- 本次涉及文件 ruff 检查通过。
- PostgreSQL SQL 编译测试确认使用 ordered-set percentile_cont 聚合。
- 全量测试:497 passed;8 个现有失败集中在邀请奖励、提现档位和代理转发等无关模块。

---------

Co-authored-by: unknown <798648091@qq.com>
Reviewed-on: #161
Co-authored-by: linkeyu <linkeyu@wonderable.ai>
Co-committed-by: linkeyu <linkeyu@wonderable.ai>
This commit was merged in pull request #161.
This commit is contained in:
2026-07-23 10:35:41 +08:00
committed by guke
parent cb8e8ccc1d
commit 77f772f47c
4 changed files with 223 additions and 45 deletions
+16
View File
@@ -4,12 +4,28 @@ from __future__ import annotations
from datetime import UTC, date, datetime
import pytest
from sqlalchemy.dialects import postgresql
from app.admin.repositories import queries
from app.db.session import SessionLocal
from app.models.comparison import ComparisonRecord
def test_postgresql_duration_summary_uses_ordered_set_aggregates() -> None:
stmt = queries._comparison_duration_aggregate_stmt(
[], "success", (0.05, 0.5, 0.95, 0.99)
)
sql = str(
stmt.compile(
dialect=postgresql.dialect(),
compile_kwargs={"literal_binds": True},
)
)
assert sql.count("percentile_cont") == 4
assert "comparison_record.status = 'success'" in sql
def test_summary_uses_only_success_durations_and_filters_beijing_date() -> None:
db = SessionLocal()
try:
+55 -2
View File
@@ -7,8 +7,12 @@ from __future__ import annotations
from datetime import UTC, date, datetime
from sqlalchemy import delete, func, select
from sqlalchemy.dialects import postgresql
from app.admin.repositories.coupon_data import coupon_data_report
from app.admin.repositories.coupon_data import (
_coupon_summary_aggregate_stmt,
coupon_data_report,
)
from app.db.session import SessionLocal
from app.models.coupon_state import CouponSession
from app.repositories.coupon_state import (
@@ -19,7 +23,12 @@ from app.repositories.coupon_state import (
def _agg_session(
trace: str, platforms, platform_success, *, status: str = "completed"
trace: str,
platforms,
platform_success,
*,
status: str = "completed",
elapsed_ms: int | None = None,
) -> CouponSession:
"""构造一条聚合测试用 session(started_date 固定 2020-01-02、app_env=prod,不 commit)。"""
return CouponSession(
@@ -31,9 +40,53 @@ def _agg_session(
platform_success=platform_success,
started_at=datetime(2020, 1, 2, tzinfo=UTC),
started_date=date(2020, 1, 2),
elapsed_ms=elapsed_ms,
)
def test_postgresql_coupon_summary_uses_ordered_set_aggregates() -> None:
sql = str(
_coupon_summary_aggregate_stmt([]).compile(
dialect=postgresql.dialect(),
compile_kwargs={"literal_binds": True},
)
)
assert sql.count("percentile_cont") == 4
assert "FILTER (WHERE coupon_session.status = 'completed'" in sql
def test_coupon_duration_summary_uses_only_completed_rows() -> None:
db = SessionLocal()
try:
db.add_all([
_agg_session("duration-a", [], [], elapsed_ms=1000),
_agg_session("duration-b", [], [], elapsed_ms=3000),
_agg_session(
"duration-failed", [], [], status="failed", elapsed_ms=100_000
),
])
db.flush()
summary = coupon_data_report(
db,
date_from="2020-01-02",
date_to="2020-01-02",
app_env="prod",
)["summary"]
assert summary["started_count"] == 3
assert summary["completed_count"] == 2
assert summary["avg_elapsed_ms"] == 2000
assert summary["p5_ms"] == 1100
assert summary["p50_ms"] == 2000
assert summary["p95_ms"] == 2900
assert summary["p99_ms"] == 2980
finally:
db.rollback()
db.close()
def _make_session(db, trace_id: str, **kw) -> CouponSession:
row = CouponSession(
trace_id=trace_id,