Compare commits

..

29 Commits

Author SHA1 Message Date
marco 4bd4e66678 重构比价结果页取数据逻辑 (#195)
Reviewed-on: #195
2026-07-29 01:59:31 +08:00
linkeyu 90c6fe599a 修复:统一用户Draw信息流eCPM统计口径 (#190)
## 问题

业务收益详情的平均 Draw eCPM 仅平均成功发奖记录,会排除未发奖的真实展示,导致数值系统性偏高,且与广告收益页口径不一致。

## 修复

- `feed_avg_ecpm` 改为从 `ad_ecpm_record` 的全部 `draw/feed` 实际展示计算
- 成功发奖、未发奖展示均纳入,每次展示等权
- 日期、正式/测试环境、业务代码位、领券/比价场景支持与广告收益页对齐
- 奖励份数仍基于成功发奖表,不混用展示数据源
- 复用广告收益报表的业务代码位集合

## 线上数据复算

2026-07-25、正式业务、用户 #33:

- 旧口径(只看成功发奖):`29.9117 元/千次`
- 新口径(333 次真实展示):`19.9926 元/千次`
- 新值与广告收益报表一致

## 验证

- 新增成功/未发奖、场景、环境、业务代码位回归用例
- `tests/test_admin_read.py` + `tests/test_admin_ad_revenue_scope.py`:29 项全通过
- Ruff 改动文件检查通过

## 上线顺序

本 PR 需先于管理后台配套 PR 上线。

---------

Co-authored-by: guke <guke@wonderable.ai>
Co-authored-by: unknown <798648091@qq.com>
Reviewed-on: #190
Co-authored-by: linkeyu <linkeyu@wonderable.ai>
Co-committed-by: linkeyu <linkeyu@wonderable.ai>
2026-07-28 17:58:23 +08:00
linkeyu e529112a90 修复中途退出比价的 LLM 成本回填 (#191)
## 问题

比价记录进入中途退出后未触发 LLM 成本回填,周期补偿也未扫描 cancelled,导致实际已有 LLM 调用的记录长期显示成本、LLM、TOKEN 为空。

## 修改

- finalize 落库后立即追加 LLM 成本回填
- 周期补偿范围加入 cancelled
- 保持无有效调用和全调用失败记录不伪造成本
- 增加即时回填和周期补偿回归测试

## 验证

- ruff 检查通过
- 相关测试 28 项通过
- 全仓 626 项通过;主干既有失败已在未修改的 origin/main 复现

---------

Co-authored-by: unknown <798648091@qq.com>
Reviewed-on: #191
Co-authored-by: linkeyu <linkeyu@wonderable.ai>
Co-committed-by: linkeyu <linkeyu@wonderable.ai>
2026-07-28 17:57:52 +08:00
guke 50da718e35 比价记录失败卡展示具体原因(新增 fail_reason) (#189)
失败记录不再一律「网络开小差」:新增记录级 fail_reason 派生列——information
具体则直出,笼统则从 platform_results 救出业务原因(找不到店/菜、未起送、打烊、
单点不配送等),纯系统失败为 None → 端侧品牌兜底。store_closed/no_delivery 被
pricebot 漏成 status=failed 的按 reason 补判,打烊脏店名统一简短模板。接入
harvest_done 与灰度期 upsert_record 两条写路径。

- models: comparison_record.fail_reason 列
- repositories: _derive_fail_display + 补判/清洗 helper,两条写路径接入
- schemas: ComparisonRecordOut 暴露 fail_reason
- alembic: 加列 + 回填老 specific 失败记录
- tests: _derive_fail_display 单测(8 例)+ harvest 失败落库集成测试

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: guke <guke@autohome.com.cn>
Reviewed-on: #189
2026-07-28 14:04:36 +08:00
guke ed76820e97 feat(sms): 短信验证码可切换多 provider(极光/阿里云/创蓝),默认极光零改动 (#188)
读完了两份 spec + 分派器,这分支的全貌清晰了。下面是可直接粘进 Gitea PR 的 MR 说明(标题 + 正文):

建议标题

feat(sms): 短信验证码可切换多 provider(极光/阿里云/创蓝),默认极光零改动
MR 正文(复制以下 markdown):

## 概述
把原单文件 `app/integrations/sms.py` 重构为**可切换 provider 包** `app/integrations/sms/`,在保留极光(默认、行为零改动)的基础上,新增两家验证码短信 provider:
- **阿里云号码认证 dypns**(Mode A:阿里云生成/存储/校验验证码,核验免费)
- **创蓝云智 253**(Mode B:本服务自管码,httpx 直连 + HMAC 签名)

Provider 由 `SMS_PROVIDER` 按调用实时选择,默认 `jiguang`。短信=花钱 + 登录关键路径,故新 provider **opt-in、可灰度、秒级回退**,默认路径零变更。

## 为什么
现有极光路径本地内存存码(多 worker 不共享,已是技术债),且单一供应商无法灰度/切换。引入 provider 抽象后:阿里云托管码可消除存码债,创蓝作为备选降低单点依赖,三家随配置切换与回退。

## 改动内容

**架构(`app/integrations/sms/`)**
| 文件 | 说明 |
|---|---|
| `__init__.py` | 对外仍暴露 `send_code/verify_code/SmsError`(auth 导入不变);按 `SMS_PROVIDER` **每次调用**分派;未知值回退 `jiguang` |
| `base.py` | `SmsError`(status_code→HTTP) + provider 无关的 `mock_verify` |
| `jiguang.py` | 原 `sms.py` 逻辑**原样迁入**,行为零改动(git 识别为 rename) |
| `aliyun.py` | 新增,Mode A:`SendSmsVerifyCode` + `CheckSmsVerifyCode`,惰性加载 SDK |
| `chuanglan.py` | 新增,Mode B:自管码 + `tpl/send` + HMAC 签名 |

**两种验证码模式**
- Mode A(阿里云):不本地存码,阿里云 `##code##` 托管生成+校验;本地仅留 per-phone 失败计数防爆破。
- Mode B(极光/创蓝):`secrets` 生成 N 位 → 进程内存 → 供应商只下发;本地一次性校验 + 失败 N 次作废。创蓝**复制**极光存码机器(不重构极光,零回归风险)。

**配置(`config.py` + `.env.example`)**
- `SMS_PROVIDER = jiguang | aliyun | chuanglan`(默认 jiguang)
- `ALIYUN_SMS_*`(AK/签名/模板/方案名/时长…) + `aliyun_sms_configured` 门控
- `CHUANGLAN_SMS_*`(账号/密码/模板/签名/endpoint…) + `chuanglan_sms_configured` 门控
- 复用现有 `SMS_MOCK / SMS_CODE_LENGTH / SMS_CODE_TTL_SEC / SMS_SEND_INTERVAL_SEC / SMS_MAX_VERIFY_ATTEMPTS`
- 切到某 provider 却未配齐 → `send_code` 抛 `SmsError(503)`,不静默

**auth.py(最小改动)**
- `verify_code` 现在可能抛 `SmsError`(阿里云降级 503)→ `sms_login`、`wechat_bind_phone_sms` 两处各包 `try/except SmsError → HTTPException`,与 `send_code` 现有写法一致。

**依赖**
- `pyproject.toml` 增 `alibabacloud_dypnsapi20170525`(仅阿里云 provider 惰性 import;jiguang/chuanglan 不加载)。创蓝零新依赖(httpx + 标准库)。

**测试**
- 新增 `test_sms_aliyun.py` / `test_sms_chuanglan.py`(均 monkeypatch 网络接缝,不发真短信) + `test_sms_dispatch.py`(分派/回退)。
- `test_auth.py` 相应更新。
- 现有测试走 `SMS_MOCK=true` 在分派层短路,不受影响。

**文档**
- 设计 spec:`docs/superpowers/specs/2026-07-25-aliyun-sms-verify-design.md`、`2026-07-26-chuanglan-sms-verify-design.md`
- 接口调研:`docs/integrations/aliyun/*`、`docs/integrations/chuanglan/tpl-send.md`、`docs/integrations/sms.md`

## 兼容性 & 回退
- **默认 `SMS_PROVIDER=jiguang`,线上行为与现状完全一致**;不改极光逻辑、不动 API 层频控与测试账号短路。
- 切阿里云/创蓝仅改环境变量,出问题秒切回极光;未知 `SMS_PROVIDER` 一律回退极光,防误配打挂登录。

---------

Co-authored-by: guke <guke@autohome.com.cn>
Reviewed-on: #188
2026-07-28 09:20:57 +08:00
guke f05dd1cf74 docs(applog): 客户端运行日志批量上报→落文件→SLS 采集 设计(spec) (#187)
Co-authored-by: guke <guke@autohome.com.cn>
Reviewed-on: #187
2026-07-27 18:00:23 +08:00
linkeyu b5962464e8 为比价记录补充是否下单状态 (#184)
## 改动说明
- 后台比价记录列表与详情增加 ordered 字段
- 按当前页批量查询真实下单记录,避免逐行查询
- 判定口径与 C 端一致:同一用户、同一店铺且 source=compare
- demo 数据不计为真实下单
- 增加列表和详情接口回归测试

## 验证
- tests/test_admin_read.py:19 项通过

---------

Co-authored-by: unknown <798648091@qq.com>
Reviewed-on: #184
Co-authored-by: linkeyu <linkeyu@wonderable.ai>
Co-committed-by: linkeyu <linkeyu@wonderable.ai>
2026-07-27 17:33:09 +08:00
linkeyu 36ce18a250 修复比价记录机型与 ROM 版本展示 (#183)
## 改动说明
- 比价记录列表和详情接口补充可读机型名
- 列表接口返回 ROM 大版本
- 保留原始设备编码,方便排查
- 增加列表与详情接口回归测试

## 验证
- 后端测试:20 项通过

---------

Co-authored-by: guke <guke@wonderable.ai>
Co-authored-by: unknown <798648091@qq.com>
Reviewed-on: #183
Co-authored-by: linkeyu <linkeyu@wonderable.ai>
Co-committed-by: linkeyu <linkeyu@wonderable.ai>
2026-07-27 16:38:56 +08:00
linkeyu 58d609e5d2 修复提现审核汇总按来源和状态统计 (#185)
## 问题

邀请提现页的汇总接口未按提现来源过滤,导致页签数字、顶部“待审核/待审核金额”与邀请提现细则不一致;同时接口缺少已到账、已拒绝和全部的历史总数。

## 改动

- 汇总接口支持并校验 `source=coin_cash|invite_cash`
- 页签数量及顶部待审核数量、金额统一按提现来源过滤
- 增加 `success_count`、`rejected_count` 和 `total_count`
- 保留今日到账/拒绝指标,并同步按来源过滤
- 增加逐来源、逐状态校验“汇总数 = 列表 total”的回归测试
- 增加“顶部待审核金额 = 当前来源待审核细则金额之和”的回归测试

## 验证

- 提现审核专项测试通过
- 后台读取接口测试:19 项通过
- Ruff(排除主干既有规则告警)通过
- 线上数据库只读复核:其他提现与邀请提现的待审核数量、金额已按来源分别核算

---------

Co-authored-by: unknown <798648091@qq.com>
Reviewed-on: #185
Co-authored-by: linkeyu <linkeyu@wonderable.ai>
Co-committed-by: linkeyu <linkeyu@wonderable.ai>
2026-07-27 16:26:02 +08:00
guke 8e01ae0bf9 修复:ComparisonResultIn 补 skipped_dish_count(逐平台缺菜数落库) (#186)
pricebot 把逐平台缺菜数冗余进 comparison_results 每行,但上报入参 schema
ComparisonResultIn 未声明该字段,model_dump() 会在 POST 路径静默丢弃,
记录页三平台网格拿不到逐格「缺少 X 个菜品」。显式声明补齐 + 回归测试。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: guke <guke@autohome.com.cn>
Reviewed-on: #186
2026-07-27 16:13:54 +08:00
linkeyu 7e17df4130 完善提现审核与用户风险管理 (#175)
## 改动内容
- 拆分邀请提现与其他提现的筛选、统计和详情数据口径
- 增加人工高风险标识、备注、权限和数据库迁移
- 完善微信失败原因查单、退款兜底与审核操作
- 移除人工处理超时订单接口和提现账本校验功能
- 补充提现审核、邀请详情、风险操作与微信失败原因回归测试

## 验证
- 相关后端测试:82 passed
- Ruff F/I:通过
- 新数据库迁移 upgrade/downgrade/upgrade:通过

## 说明
- 本地 seed mock 脚本未提交

---------

Co-authored-by: guke <guke@wonderable.ai>
Co-authored-by: unknown <798648091@qq.com>
Reviewed-on: #175
Co-authored-by: linkeyu <linkeyu@wonderable.ai>
Co-committed-by: linkeyu <linkeyu@wonderable.ai>
2026-07-27 16:03:31 +08:00
linkeyu 46f68b88e3 修复:统一广告收益分类 eCPM 口径 (#178)
## 本次改动

- Draw 信息流 eCPM 合并 `draw + 历史 feed`
- 看视频 eCPM 合并 `reward_video + withdrawal_video`
- 直接按每次真实展示的 SDK eCPM 加权,不再由发奖状态修正后的收益反推
- 增加经营分类聚合接口字段和专项测试

## 验证

`tests/test_admin_ad_revenue_scope.py`:3 passed。

---------

Co-authored-by: unknown <798648091@qq.com>
Reviewed-on: #178
Co-authored-by: linkeyu <linkeyu@wonderable.ai>
Co-committed-by: linkeyu <linkeyu@wonderable.ai>
2026-07-27 16:02:40 +08:00
linkeyu 1226bc8365 修复比价记录平均 TOKEN 成本采集与回填 (#182)
## 问题原因

- 新版客户端通过服务端 harvest 落库,但完成路径没有触发 LLM 调用明细与 TOKEN 成本回填。
- 内部共享密钥不一致或 PriceBot 实例切换后,拉取失败只留下空值,后续没有自动补偿。

## 本次改动

- harvest 完成后立即异步回填 LLM 调用、TOKEN 数及成本快照。
- 抽取统一、幂等的成本回填服务,并增加定时补偿 worker。
- 增加 PriceBot 内部鉴权预检、错误日志和多实例查找兜底。
- 仅回填当前价格配置生效后的终态记录,避免用现价误算更早历史数据。
- 补充环境配置、部署说明和单元测试。

## 验证

- 相关测试:36 passed。
- 静态检查通过,diff check 通过。
- 本地页面显示 ¥0.0139,与数据库精确均值 0.013916 的四舍五入结果一致。

## 上线注意

部署时需确保 app-server 与 PriceBot 的 INTERNAL_API_SECRET 完全一致并重启两个服务;worker 启动后会自动补齐符合条件的历史空值。

---------

Co-authored-by: guke <guke@wonderable.ai>
Co-authored-by: unknown <798648091@qq.com>
Reviewed-on: #182
Co-authored-by: linkeyu <linkeyu@wonderable.ai>
Co-committed-by: linkeyu <linkeyu@wonderable.ai>
2026-07-27 15:51:19 +08:00
linkeyu 22a1105000 修复用户反馈机型与系统版本补全 (#181)
## 改动说明
- 反馈列表按同一用户、同一设备编码和反馈提交时间,补全可读机型、厂商及 ROM 大版本
- 补充常见线上机型编码映射
- 更新本地 mock 数据并增加时间边界回归测试

## 验证
- 反馈相关测试:3 项通过
- Ruff 检查通过

---------

Co-authored-by: unknown <798648091@qq.com>
Reviewed-on: #181
Co-authored-by: linkeyu <linkeyu@wonderable.ai>
Co-committed-by: linkeyu <linkeyu@wonderable.ai>
2026-07-27 13:33:55 +08:00
guke b2a528eba1 fix(comparison): 派生 best 排除缺菜店, 避免虚低价当"最低价" (#176)
pricebot comparison_results[].rank 是纯价格排序(含缺菜店), server 派生 best
若照单全收, 会把缺菜(漏菜)店的虚低总价当 best_price → 记录页戴"最低"红框 +
算出虚假省额。

- _derive / _derive_from_results 派生 best 时按 platform_results[pid].
  skipped_dish_count 排除缺菜店; 源平台永远全菜, 全目标缺菜时回落到源
  (is_source_best、saved=0), 不虚报省额。
- platform_results 内层结构宽松(老客户端透传可伪造), _is_short 用
  isinstance 兜底, 值非 dict 时按"不缺菜"处理, 不打 500。
- harvest_done 传入 done_params.platform_results; 不传→纯 rank/price 老行为不变。
- 新增纯函数测试: 排除缺菜 / 全缺菜回落源 / 不传保持老行为 / 内层非 dict 不崩,
  覆盖 _derive 与 _derive_from_results 两条路径。

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: guke <guke@autohome.com.cn>
Reviewed-on: #176
2026-07-27 10:35:15 +08:00
linkeyu cdd49c6421 修复:补齐收益明细广告网络来源 (#179)
## 本次改动

- 激励视频发奖记录按用户与 `ad_session_id` 回填展示侧 ADN
- 信息流明细保留每条发奖自身上报的 ADN 与底层代码位
- 历史记录仅在 `用户 + trace_id + eCPM` 候选网络唯一时安全回填
- 多网络候选保持空值,避免错误归因

## 验证

专项测试 4 passed,覆盖不同 ADN 明细及唯一/歧义回填场景。

---------

Co-authored-by: guke <guke@wonderable.ai>
Co-authored-by: unknown <798648091@qq.com>
Reviewed-on: #179
Co-authored-by: linkeyu <linkeyu@wonderable.ai>
Co-committed-by: linkeyu <linkeyu@wonderable.ai>
2026-07-27 10:24:32 +08:00
linkeyu 775a503d6f 修复广告收益用户详情加载失败 (#180)
## 问题原因
- 奖励统计接口用完整 User ORM 判断用户存在,滚动发布或表结构未同步时会因无关字段导致 500
- 金币记录合并广告与签到数据后直接排序,PostgreSQL 中 aware/naive datetime 混排会抛异常

## 修复内容
- 新增只投影 user.id 的用户存在性检查,保持不存在用户返回 404
- 金币记录排序前统一转换为 aware UTC 排序键
- 增加旧表结构投影与混合时区回归测试

## 验证结果
- tests/test_admin_read.py:18 项全部通过
- 线上只读数据库回归:近期 7 个活跃用户统计与金币明细全部正常返回;用户 #33 返回 1514 条记录
- 语法/未定义引用检查通过
- 全量测试 532 通过、9 失败;失败均在未修改的 origin/main 基线上复现,和本 PR 无关

---------

Co-authored-by: unknown <798648091@qq.com>
Reviewed-on: #180
Co-authored-by: linkeyu <linkeyu@wonderable.ai>
Co-committed-by: linkeyu <linkeyu@wonderable.ai>
2026-07-27 10:15:54 +08:00
linkeyu 7bf04f2655 功能:新增风控监控与处置能力 (#174)
## 修改内容
- 新增短信、一键登录、比价三类风控事件与规则聚合
- 新增管理员忽略、封禁、解封、重置报警和阈值配置接口
- 风险列表返回当前有效限制的 restriction_id,供后台已封禁视图解除
- 在短信/一键登录、比价、任务领奖、提现链路接入风险记录与限制
- 新增通用行为流水、风险事件、主体限制模型及 Alembic 迁移
- 新增本地演示数据脚本与风控测试

## 验证
- ruff check:通过
- Alembic 全新 SQLite upgrade head / downgrade -1:通过
- 风控测试:通过,覆盖封禁列表 restriction_id 与解除链路
- 全量测试:532 passed,8 failed;其中 7 项在干净 origin/main 独立复现,另 1 项独立复跑通过,未发现本分支新增回归

---------

Co-authored-by: unknown <798648091@qq.com>
Reviewed-on: #174
Co-authored-by: linkeyu <linkeyu@wonderable.ai>
Co-committed-by: linkeyu <linkeyu@wonderable.ai>
2026-07-25 17:56:23 +08:00
marco 5f6593eaf2 fix(alembic): 缩短 withdraw 迁移 revision id 至 varchar(32) 内
drop_withdraw_active_unique_index (33 字符) 超过 alembic_version.version_num
的 varchar(32) 上限,生产 alembic 写版本号时 StringDataRightTruncation 报错、
部署中断。改短为 drop_withdraw_active_uniq_idx (29 字符):迁移逻辑(drop_index)
一字不变、无下游引用、仍为唯一 head。

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 20:32:38 +08:00
guke e11f506e1e docs: 提现允许在途时继续提交申请 设计spec (#173)
## 摘要
取消「同一用户同时仅一笔在途提现」限制:已有 reviewing/pending 提现单时可继续发起新申请。

- 删应用层在途单检查(WithdrawTooFrequentError)+ 删 DB 分区唯一索引 ux_withdraw_order_user_active(含迁移)
- 清理失效死代码;IntegrityError 兜底瘦身为仅处理 out_bill_no 幂等
- 既有约束不变:建单先扣款(防超提)、coin_cash 每日档位次数、out_bill_no 幂等、解绑退款、admin 审核/对账均按单号维度

## 测试
- 新增:多笔在途并存放行(coin_cash & invite_cash)、第二笔仅受余额约束(409 现金余额不足)
- 迁移 upgrade→downgrade→upgrade 回环验证
- 提现域全绿(test_withdraw / test_invite_cash_withdraw / test_withdraw_ledger_check)

## 注意
- 客户端:每次提交需生成新的 out_bill_no;未开免确认时多笔 pending 各返回一个微信确认页,App 需能处理多笔待确认
- 无并发硬上限(产品拍板):coin_cash 由每日档位次数天然封顶,invite_cash 仅受余额约束

---------

Co-authored-by: guke <guke@autohome.com.cn>
Reviewed-on: #173
2026-07-24 16:40:57 +08:00
guke addc30817f 修复:coupon_claim_event 回填按 (trace_id, coupon_id) 去重,避免上线迁移唯一约束冲突 (#172)
旧回填 INSERT ... SELECT 从 coupon_claim_record 整表 1:1 复制进 coupon_claim_event。
两表唯一键不同:源表按 (device_id, coupon_id, claim_date) 去重,trace_id 可空且不在
唯一键里;新表按 (trace_id, coupon_id) 唯一。一次会话的 /step 帧跨零点(Asia/Shanghai)
时,同一 (trace_id, coupon_id) 会落在相邻两天两行,复制时撞 uq_coupon_claim_event_trace_coupon,
整个迁移事务回滚,线上发车中断(线上命中 (1581db0e..., mt_banjia_zhoumo))。

回填改为按 (trace_id, coupon_id) 只取 id 最大(最近写入)的一行,SQLite/PostgreSQL 通用。
运行期写入路径(record_claims 按 (trace_id, coupon_id) upsert)本就无重复,仅此一次性回填未做防御。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: guke <guke@autohome.com.cn>
Reviewed-on: #172
2026-07-24 15:12:20 +08:00
linkeyu e165fcfc5e 修复:自动查单关闭不再标记为支付异常 (#170)
## 改动说明

- 自动查单关闭时不再写入微信提现配置异常列表。
- 保留自动查单状态字段、部署开关和运营开关,不改变 worker 实际行为。
- 明确自动查单属于非阻断运维能力,避免与“无法微信打款”混淆。

## 验证

- `pytest tests/test_withdraw_ledger_check.py tests/test_admin_config.py -q`
- 8 项测试全部通过。
- 本地管理 API 健康检查确认不再返回自动查单关闭问题。

---------

Co-authored-by: guke <guke@wonderable.ai>
Co-authored-by: unknown <798648091@qq.com>
Reviewed-on: #170
Co-authored-by: linkeyu <linkeyu@wonderable.ai>
Co-committed-by: linkeyu <linkeyu@wonderable.ai>
2026-07-24 14:51:31 +08:00
zuochenyong 3f7b5167fa 功能:新手引导视频 + 美团券首页分页索引 (#167)
Co-authored-by: guke <guke@wonderable.ai>
Co-authored-by: 左辰勇 <exinglang@gmail.com>
Reviewed-on: #167
Co-authored-by: zuochenyong <zuochenyong@wonderable.ai>
Co-committed-by: zuochenyong <zuochenyong@wonderable.ai>
2026-07-24 14:48:40 +08:00
linkeyu 9e88ca72d3 修复提现审核详情点击用户后提示操作失败 (#171)
## 问题现象

在“提现审核”页点击用户所在行后,提现单主详情可以打开,但用户统计和金币记录区域为空,页面连续提示“操作失败”。

## 原因说明

打开抽屉时前端会继续请求两个子接口:

- `/admin/api/users/{user_id}/reward-stats`
- `/admin/api/users/{user_id}/coin-records`

这两个接口原来都使用 `select(AdRewardRecord)` 加载完整 ORM 对象。SQLAlchemy 会把模型映射的所有列自动展开到 SQL 中,其中包括后来新增的 `boost_round_id`。当旧本地数据库或滚动发布中的数据库尚未补齐该列时,即使提现详情本身完全不使用这个字段,查询仍会报 `no such column: ad_reward_record.boost_round_id`,两个接口均返回 500。

前端的统一错误处理只会展示响应 JSON 中字符串类型的 `detail`;该 500 返回的是普通 `Internal Server Error`,因此最终回退成通用文案“操作失败”。本地前端开启了 React Strict Mode,初始化副作用在开发环境会执行两次,所以两个失败接口会形成截图中的四条“操作失败”提示。

## 修复方案

- 用户奖励统计只查询实际需要的 `ecpm_raw`、`coin` 等字段。
- 金币记录只查询页面展示、排序所需字段。
- 同步缩小信息流广告和签到记录的字段投影,避免将来新增无关 ORM 列再次拖垮详情页。
- 增加 SQL 级回归测试:主动拦截任何包含 `ad_reward_record.boost_round_id` 的详情查询,并验证两个接口仍返回 200。

该改动不会改变统计口径或返回结构。数据库迁移仍应正常执行;这里增加的是旧库及滚动发布期间的向后兼容保护。

## 验证结果

- `pytest tests/test_admin_read.py -q`:17 passed
- 新增回归测试覆盖 `reward-stats` 与 `coin-records`
- `git diff --check`:通过
- 本地实际提现用户接口验证:两个接口均返回 200

---------

Co-authored-by: unknown <798648091@qq.com>
Reviewed-on: #171
Co-authored-by: linkeyu <linkeyu@wonderable.ai>
Co-committed-by: linkeyu <linkeyu@wonderable.ai>
2026-07-24 14:04:04 +08:00
linkeyu 21a4d0af5b 后台审核新增批量处理接口 (#164)
## 改动
- 新增低价审核与用户反馈的批量通过、批量拒绝接口
- 单条仍保持独立事务、审计、发奖和通知;单项失败不影响同批其它记录
- 批量响应返回每条记录的成功状态或失败原因,供前端保留失败项重试
- 反馈审核补充行锁,降低并发重复发奖风险

## 验证
- `ruff check`(相关路由、Schema、测试)
- `pytest tests/test_admin_write.py -q`:22 passed

---------

Co-authored-by: unknown <798648091@qq.com>
Reviewed-on: #164
Co-authored-by: linkeyu <linkeyu@wonderable.ai>
Co-committed-by: linkeyu <linkeyu@wonderable.ai>
2026-07-24 12:04:41 +08:00
guke f7a7a49281 fix(withdraw): 免确认授权已开启判定加 authorization_id 非空,与打款一致 (#168)
免确认授权已开启判定加 authorization_id 非空,与打款一致

---------

Co-authored-by: guke <guke@autohome.com.cn>
Reviewed-on: #168
2026-07-24 11:17:09 +08:00
linkeyu 71aef455f4 修复:签到膨胀金币归入看视频分类 (#169)
## 修改内容

- 将历史 `signin_boost` 金币从“常规任务金币”分类移出。
- 将 `signin_boost` 与 `reward_video/ad_reward` 一起计入“看视频金币”。
- 保留独立的 `signin_boost_coin_total` 历史审计字段。
- 增加不重不漏回归测试,确认分类调整前后本期发放总额保持不变。

## 本地验证

- `tests/test_admin_read.py`:16 项通过。
- Ruff 与 `git diff --check`:通过。
- 全量后端测试:505 项通过;8 项为 `main` 现有无关失败。

---------

Co-authored-by: unknown <798648091@qq.com>
Reviewed-on: #169
Co-authored-by: linkeyu <linkeyu@wonderable.ai>
Co-committed-by: linkeyu <linkeyu@wonderable.ai>
2026-07-24 11:16:38 +08:00
linkeyu 66527f6cdc 修复:领券单券成功率按任务独立统计 (#166)
## 修改内容

- 新增按 `trace_id + coupon_id` 幂等的逐次单券事件表。
- 保留每日资产记录,同时独立保存每次领券事件,避免同设备同日重跑串场。
- 后台逐场成功率和单券明细改为读取逐次事件。
- 增加历史数据回填迁移、本地 mock 脚本和回归测试。

## 验证结果

- 领券相关测试:22 项通过。
- 全新 SQLite 数据库执行 `alembic upgrade head`:通过。
- 全量测试:508 项通过;另外 8 项为现有无关失败。

---------

Co-authored-by: unknown <798648091@qq.com>
Reviewed-on: #166
Co-authored-by: linkeyu <linkeyu@wonderable.ai>
Co-committed-by: linkeyu <linkeyu@wonderable.ai>
2026-07-24 11:15:09 +08:00
linkeyu b4a2a8c31d 功能:限制每位用户每天最多发起 100 次比价 (#165)
## 变更内容

- 登录用户按北京时间自然日计算比价发起次数,每人每天最多 100 次。
- 第 101 次起返回 HTTP 429,并提示“今日已比价超过100次,请明天再试”。
- 同一 trace_id 的网络重试按幂等处理,不会重复计数。
- 使用用户行锁串行化同一账号的并发请求,避免并发突破上限。
- 复用现有 comparison_record 的 running 记录,无需新增数据库迁移。

## 本地验证

- 比价额度专项测试 11 项通过。
- 本次修改涉及文件的 Ruff 检查通过。
- 已覆盖未登录、重复 trace_id、跨自然日、第 100 次放行及第 101 次拒绝。

---------

Co-authored-by: CodexSandboxOffline <798648091@qq.com>
Reviewed-on: #165
Co-authored-by: linkeyu <linkeyu@wonderable.ai>
Co-committed-by: linkeyu <linkeyu@wonderable.ai>
2026-07-24 11:14:31 +08:00
136 changed files with 13860 additions and 994 deletions
+40 -73
View File
@@ -27,66 +27,44 @@ JG_PRIVATE_KEY_PATH=./secrets/jverify_rsa_private.pem
JG_VERIFY_ENDPOINT=https://api.verification.jpush.cn/v1/web/loginTokenVerify
JG_REQUEST_TIMEOUT_SEC=15
# ===== 厂商直推(无障碍保护存活告警 + 消息中心 13 类通知)=====
# 敏感密钥只放 .env / 服务器环境变量,不要提交到 git。
# 各厂商配置状态可随时 GET /api/v1/push/vendors 查看(缺哪些键一目了然)。
ANDROID_PACKAGE_NAME=com.jishisongfu.shaguabijia
PUSH_REQUEST_TIMEOUT_SEC=15
PUSH_TIME_TO_LIVE_SEC=86400
HONOR_PUSH_APP_ID=
HONOR_PUSH_CLIENT_ID=
HONOR_PUSH_CLIENT_SECRET=
HONOR_PUSH_TOKEN_ENDPOINT=https://iam.developer.honor.com/auth/token
HONOR_PUSH_SEND_ENDPOINT_TEMPLATE=https://push-api.cloud.honor.com/api/v1/{app_id}/sendMessage
# 华为 Push Kit:AGC 控制台 → 项目设置 → 常规 → 应用,AppId + AppSecret
HUAWEI_PUSH_APP_ID=
HUAWEI_PUSH_APP_SECRET=
HUAWEI_PUSH_TOKEN_ENDPOINT=https://oauth-login.cloud.huawei.com/oauth2/v3/token
HUAWEI_PUSH_SEND_ENDPOINT_TEMPLATE=https://push-api.cloud.huawei.com/v1/{app_id}/messages:send
VIVO_PUSH_APP_ID=
VIVO_PUSH_APP_KEY=
VIVO_PUSH_APP_SECRET=
VIVO_PUSH_AUTH_ENDPOINT=https://api-push.vivo.com.cn/message/auth
VIVO_PUSH_SEND_ENDPOINT=https://api-push.vivo.com.cn/message/send
# vivo 未上架测试时可用 push_mode=1; 上架正式推送改为 0。
VIVO_PUSH_MODE=1
VIVO_PUSH_NOTIFY_TYPE=4
VIVO_PUSH_CATEGORY=DEVICE_REMINDER
XIAOMI_PUSH_APP_SECRET=
XIAOMI_PUSH_SEND_ENDPOINT=https://api.xmpush.xiaomi.com/v3/message/regid
XIAOMI_PUSH_CHANNEL_ID=
XIAOMI_PUSH_TEMPLATE_ID=
XIAOMI_PUSH_TEMPLATE_TITLE=
XIAOMI_PUSH_TEMPLATE_DESCRIPTION=
# 可选: JSON 字符串,支持 {title}/{alert} 占位符,例如 {"title":"{title}","content":"{alert}"}
XIAOMI_PUSH_TEMPLATE_PARAM_JSON=
OPPO_PUSH_APP_KEY=
OPPO_PUSH_MASTER_SECRET=
OPPO_PUSH_AUTH_ENDPOINT=https://api.push.oppomobile.com/server/v1/auth
OPPO_PUSH_SEND_ENDPOINT=https://api.push.oppomobile.com/server/v1/message/notification/unicast
# OPPO 新消息分类(2024-11-20 后创建的应用必须携带 category;channel_id 为后台「通道ID」;
# notify_level 0=不传走默认,内容营销类仅支持 1/2)
OPPO_PUSH_CHANNEL_ID=
OPPO_PUSH_CATEGORY=
OPPO_PUSH_NOTIFY_LEVEL=0
# ===== 无障碍保护存活监控(推送 + pull 后置兜底)=====
# ===== 无障碍保护存活监控(pull 后置检测;本期不接推送)=====
HEARTBEAT_MONITOR_ENABLED=true
HEARTBEAT_TIMEOUT_MINUTES=60
HEARTBEAT_SCAN_INTERVAL_SEC=60
# ===== 短信 (mock 模式) =====
# mock = true 时,任意 6 位数字均通过,且 /sms/send 不真发短信(只 log)。
# 后续接阿里云/腾讯云短信时,改成 false 并填供应商相关 key。
# mock = true 时,任意 6 位数字均通过,且 /sms/send 不真发短信(只 log)。生产改 false。
SMS_MOCK=true
SMS_CODE_TTL_SEC=300
SMS_SEND_INTERVAL_SEC=60
# ===== 短信提供商(可切换:jiguang 默认 / aliyun 阿里云号码认证 / chuanglan 创蓝云智)=====
# jiguang :本服务生成验证码,极光 REST 只负责下发,本地内存校验(复用上面极光 JG_* 凭证)。
# aliyun :阿里云 dypns 号码认证,阿里云生成+下发+校验(Mode A,核验免费);缺凭证时 /sms/* 返 503。
# 需在阿里云号码认证控制台开通「融合认证」,并使用系统赠送签名 + 赠送模板。
# chuanglan:创蓝云智(253)模板短信,本服务生成码、创蓝只下发、本地校验(Mode B,与极光同);缺凭证 503。
# 用 YZM 前缀验证码账号;服务器出网 IP 需在创蓝控制台加白名单(否则 117)。见 docs/integrations/chuanglan/tpl-send.md。
SMS_PROVIDER=jiguang
ALIYUN_SMS_ACCESS_KEY_ID=
ALIYUN_SMS_ACCESS_KEY_SECRET=
ALIYUN_SMS_SIGN_NAME=
ALIYUN_SMS_TEMPLATE_CODE=
# 方案名:留空=默认方案;若填,发码与校验须一致(本服务已共用同一配置项,不会不匹配)。
ALIYUN_SMS_SCHEME_NAME=
ALIYUN_SMS_ENDPOINT=dypnsapi.aliyuncs.com
ALIYUN_SMS_CODE_LENGTH=6
ALIYUN_SMS_VALID_TIME_SEC=300
ALIYUN_SMS_INTERVAL_SEC=60
ALIYUN_SMS_TIMEOUT_SEC=15
# --- 创蓝云智(253)---
CHUANGLAN_SMS_ACCOUNT=
CHUANGLAN_SMS_PASSWORD=
CHUANGLAN_SMS_TEMPLATE_ID=1022457679
# 短信签名文案【品牌】;模板已关联签名则留空。
CHUANGLAN_SMS_SIGNATURE=
CHUANGLAN_SMS_ENDPOINT=https://smssh.253.com/msg/sms/v2/tpl/send
CHUANGLAN_SMS_TIMEOUT_SEC=10
# ===== 测试账号(release 包全流程联调用)=====
# 配一个固定测试手机号,专供无 SIM 卡 / 不走一键登录时打通全流程:该号登录【免短信验证码】
# (real 模式下也跳过校验)、每次登录【都重走新手引导】,并有【每日登录上限】防被人猜到号后脚本刷。
@@ -113,13 +91,6 @@ JD_UNION_APP_SECRET=
JD_UNION_SITE_ID=
JD_UNION_AUTH_KEY=
# 美团 + 京东订单每天北京时间 05:00 自动对账;按更新时间回拉近 3 天,重叠防漏单并刷新状态。
# 手动对账按钮不受该开关影响。通常保持开启;临时停自动任务时设为 false。
CPS_AUTO_RECONCILE_ENABLED=true
CPS_AUTO_RECONCILE_RUN_HOUR=5
CPS_AUTO_RECONCILE_LOOKBACK_DAYS=3
CPS_AUTO_RECONCILE_CHECK_INTERVAL_SEC=60
# ===== Pricebot 上游 (领券/比价业务透传目标) =====
# 客户端调本服务的 /api/v1/coupon/step 等,我们透传到 pricebot-backend。
# 本地开发用 localhost:8000。生产部署改成内网地址(如 http://pricebot.internal:8000)。
@@ -193,18 +164,14 @@ PANGLE_REPORT_SECURITY_KEY=
PANGLE_REPORT_SITE_ID_PROD=5830519
PANGLE_REPORT_SITE_ID_TEST=5832303
# ===== 可观测(OpenObserve 接口指标)=====
# 采集每个接口 QPS + 耗时 + 错误率,批量直采到 OpenObserve(本地 Docker,见 deploy/openobserve/)。
# 默认关;开启需 ENABLED=true 且填 USER/PASSWORD(与 docker-compose 里 root 账号一致)。
# 未开/缺凭证 → 中间件透传、worker 不启动,整套 no-op,不影响业务
OBSERVE_ENABLED=false
OBSERVE_ENDPOINT=http://localhost:5080
OBSERVE_ORG=default
OBSERVE_STREAM=app_requests
OBSERVE_USER=admin@shaguabijia.local
OBSERVE_PASSWORD=Complexpass#123
# 进阶(一般不用改):攒批间隔秒 / 单批最大条数 / 有界队列上限(满则丢) / 上报超时秒
OBSERVE_FLUSH_INTERVAL_SEC=5
OBSERVE_BATCH_MAX=200
OBSERVE_QUEUE_MAX=10000
OBSERVE_TIMEOUT_SEC=5
# ===== 客户端运行日志上报(POST /api/v1/applog/batch)=====
# 客户端批量上报的 App 运行日志逐条落到独立滚动文件 logs/app-client.log,供阿里云 Logtail
# 采进【独立 SLS logstore】(与服务日志 app-server.log 分开;滚动机制相同,trace_id 可跨层检索)。
# 全部有默认值,不填即用默认(定义见 app/core/client_log.py 与 app/api/v1/applog.py)
# CLIENT_LOG_FILE=logs/app-client.log # 落盘路径
# CLIENT_LOG_MAX_BYTES=20971520 # 单文件 20MB 滚动
# CLIENT_LOG_BACKUP_COUNT=10 # 保留 10 个 → ~200MB 缓冲(给 Logtail 断线留余量)
# CLIENT_LOG_SERVICE_NAME=app-client # 输出行 service 字段
# APPLOG_MAX_BATCH=500 # 单批最大条数(超 → 422;导入期常量,改需重启)
# APPLOG_MAX_BODY_BYTES=2097152 # 请求体上限 2MB(超 → 413;运行期可调)
# APPLOG_MAX_MSG_BYTES=8192 # 单条 msg 超此字节数截断
+3
View File
@@ -49,6 +49,9 @@ secrets/*
*.log
logs/
# 本地 admin server(端口 8771)Windows 启动脚本,个人调试用,不入库
/run8771.bat
# Claude Code 自动持久化的权限 allowlist / 个人本地设置(会话专属,不入库)。
# 需要团队共享的 Claude 配置(commands/ 等)可单独 git add -f,不受此忽略影响。
.claude/settings.json
@@ -0,0 +1,26 @@
"""merge comparison platforms + fail_reason heads
Revision ID: 6d2309208549
Revises: comparison_platforms_col, comparison_record_fail_reason
Create Date: 2026-07-29 01:48:41.868083
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = '6d2309208549'
down_revision: Union[str, Sequence[str], None] = ('comparison_platforms_col', 'comparison_record_fail_reason')
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
pass
def downgrade() -> None:
pass
@@ -0,0 +1,47 @@
"""add platforms unified array column to comparison_record
展示模型统一数组(pricebot done.params.platforms 原样存): 每平台一行、自带
status/is_best/display, 记录页据此直接渲染, 不再靠 comparison_results + 客户端合并 + 前端派生。
纯新增列, 老记录为空 → 前端回退老 comparison_results。
Revision ID: comparison_platforms_col
Revises: user_manual_risk_fields
"""
from __future__ import annotations
from collections.abc import Sequence
import sqlalchemy as sa
from sqlalchemy.dialects import postgresql
from alembic import op
revision: str = "comparison_platforms_col"
down_revision: str | None = "user_manual_risk_fields"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
_JSON = sa.JSON().with_variant(postgresql.JSONB(), "postgresql")
def upgrade() -> None:
# 幂等: 线上为了提前给历史数据补 platforms(2026-07-29), 已手动
# `ALTER TABLE comparison_record ADD COLUMN IF NOT EXISTS platforms jsonb
# NOT NULL DEFAULT '[]'::jsonb`(与本 migration 定义一致)。列已存在时跳过,
# 否则上线 alembic upgrade head 会撞 DuplicateColumn 直接部署失败。
bind = op.get_bind()
cols = {c["name"] for c in sa.inspect(bind).get_columns("comparison_record")}
if "platforms" in cols:
return
with op.batch_alter_table("comparison_record") as batch_op:
batch_op.add_column(
sa.Column(
"platforms", _JSON, nullable=False,
server_default=sa.text("'[]'"),
)
)
def downgrade() -> None:
with op.batch_alter_table("comparison_record") as batch_op:
batch_op.drop_column("platforms")
@@ -0,0 +1,49 @@
"""comparison_record.fail_reason (失败卡展示原因)
Revision ID: comparison_record_fail_reason
Revises: user_manual_risk_fields
Create Date: 2026-07-28 12:00:00.000000
失败记录的展示原因:information 具体则=它;笼统则由写路径从 platform_results 捞出的
业务原因;纯系统失败为 None(端侧品牌兜底)。见 repositories.comparison._derive_fail_display。
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = 'comparison_record_fail_reason'
down_revision: Union[str, Sequence[str], None] = 'user_manual_risk_fields'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
with op.batch_alter_table('comparison_record', schema=None) as batch_op:
batch_op.add_column(sa.Column('fail_reason', sa.String(length=256), nullable=True))
# 回填老失败记录:information 具体的直接搬过来(笼统/系统失败留 None → 端侧品牌兜底)。
# 新记录由写路径 _derive_fail_display 落库(含 platform_results 救援/补判),不走这条。
# platform_results 只在 raw_payload 里,SQL 里不易解析,故老记录不做救援/补判(可接受:
# 老 mixed/打烊记录回退品牌兜底);具体 information 的老记录本次即可显示真实原因。
op.execute(
"""
UPDATE comparison_record
SET fail_reason = information
WHERE status = 'failed'
AND information IS NOT NULL
AND information <> ''
AND information NOT IN (
'比价过程出错,请稍后重试',
'比价出错',
'比价未完成',
'done 参数缺少可验证的目标平台结果'
)
"""
)
def downgrade() -> None:
with op.batch_alter_table('comparison_record', schema=None) as batch_op:
batch_op.drop_column('fail_reason')
+103
View File
@@ -0,0 +1,103 @@
"""add per-session coupon claim event table
Revision ID: coupon_claim_event
Revises: 8e04cc13a211
Create Date: 2026-07-23
"""
from collections.abc import Sequence
import sqlalchemy as sa
from sqlalchemy.dialects import postgresql
from alembic import op
revision: str = "coupon_claim_event"
down_revision: str | Sequence[str] | None = "8e04cc13a211"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
_JSON = sa.JSON().with_variant(postgresql.JSONB(), "postgresql")
def upgrade() -> None:
op.create_table(
"coupon_claim_event",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("trace_id", sa.String(length=64), nullable=False),
sa.Column("device_id", sa.String(length=64), nullable=False),
sa.Column("user_id", sa.Integer(), nullable=True),
sa.Column("coupon_id", sa.String(length=64), nullable=False),
sa.Column("claim_date", sa.Date(), nullable=False),
sa.Column("status", sa.String(length=24), nullable=False),
sa.Column("app_env", sa.String(length=16), nullable=True),
sa.Column("vendor", sa.String(length=48), nullable=True),
sa.Column("coupon_name", sa.String(length=128), nullable=True),
sa.Column("claimed_count", sa.Integer(), nullable=True),
sa.Column("reason", sa.String(length=255), nullable=True),
sa.Column("extra", _JSON, nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint(
"trace_id", "coupon_id",
name="uq_coupon_claim_event_trace_coupon",
),
)
op.create_index(
"ix_coupon_claim_event_date_env",
"coupon_claim_event",
["claim_date", "app_env"],
unique=False,
)
op.create_index(
op.f("ix_coupon_claim_event_app_env"),
"coupon_claim_event",
["app_env"],
unique=False,
)
op.create_index(
op.f("ix_coupon_claim_event_trace_id"),
"coupon_claim_event",
["trace_id"],
unique=False,
)
op.create_index(
op.f("ix_coupon_claim_event_user_id"),
"coupon_claim_event",
["user_id"],
unique=False,
)
# 旧表按 (device, coupon, 自然日) 去重,trace_id 可空且不在唯一键里:同一
# (trace_id, coupon_id) 可能散落在多行(如一次会话的 /step 帧跨零点,把同一张券
# 写进相邻两天)。新表按 (trace_id, coupon_id) 唯一,整表 1:1 复制会撞
# uq_coupon_claim_event_trace_coupon。回填时按 (trace_id, coupon_id) 只取 id 最大
# (最近写入)的一行。历史上已被每日去重覆盖的关联仍无法恢复。
op.execute(
"""
INSERT INTO coupon_claim_event (
trace_id, device_id, user_id, coupon_id, claim_date, status, app_env,
vendor, coupon_name, claimed_count, reason, extra, created_at, updated_at
)
SELECT
trace_id, device_id, user_id, coupon_id, claim_date, status, app_env,
vendor, coupon_name, claimed_count, reason, extra, created_at, updated_at
FROM coupon_claim_record
WHERE trace_id IS NOT NULL
AND id IN (
SELECT MAX(id)
FROM coupon_claim_record
WHERE trace_id IS NOT NULL
GROUP BY trace_id, coupon_id
)
"""
)
def downgrade() -> None:
op.drop_index(op.f("ix_coupon_claim_event_user_id"), table_name="coupon_claim_event")
op.drop_index(op.f("ix_coupon_claim_event_trace_id"), table_name="coupon_claim_event")
op.drop_index(op.f("ix_coupon_claim_event_app_env"), table_name="coupon_claim_event")
op.drop_index("ix_coupon_claim_event_date_env", table_name="coupon_claim_event")
op.drop_table("coupon_claim_event")
@@ -0,0 +1,26 @@
"""merge guide_video seq_uq and coupon_claim_event heads
Revision ID: d8dd2106e438
Revises: coupon_claim_event, guide_video_user_seq_uq
Create Date: 2026-07-24 11:52:18.290731
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = 'd8dd2106e438'
down_revision: Union[str, Sequence[str], None] = ('coupon_claim_event', 'guide_video_user_seq_uq')
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
pass
def downgrade() -> None:
pass
@@ -0,0 +1,26 @@
"""merge guide_video and main alembic heads
Revision ID: d9c03cc3ea07
Revises: 8e04cc13a211, guide_video_play_table
Create Date: 2026-07-23 22:57:40.998161
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = 'd9c03cc3ea07'
down_revision: Union[str, Sequence[str], None] = ('8e04cc13a211', 'guide_video_play_table')
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
pass
def downgrade() -> None:
pass
@@ -0,0 +1,37 @@
"""drop withdraw active-order partial unique index (allow multiple in-flight withdrawals)
Revision ID: drop_withdraw_active_uniq_idx
Revises: d8dd2106e438
Create Date: 2026-07-24 00:00:00.000000
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = "drop_withdraw_active_uniq_idx"
down_revision: Union[str, Sequence[str], None] = "d8dd2106e438"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# 取消「同一用户同一时刻仅一笔在途提现」:允许 reviewing/pending 并存。
# 仅删本索引;姊妹索引 ux_cash_transaction_withdraw_refund_ref(退款幂等)保持不动。
op.drop_index("ux_withdraw_order_user_active", table_name="withdraw_order")
def downgrade() -> None:
# 回滚重建分区唯一索引。注意:若届时某用户已有 ≥2 张在途单,重建会因唯一冲突失败——
# 属预期的回滚代价(取消限制后本就允许多单),需先人工收敛在途单再回滚。
op.create_index(
"ux_withdraw_order_user_active",
"withdraw_order",
["user_id"],
unique=True,
sqlite_where=sa.text("status IN ('reviewing', 'pending')"),
postgresql_where=sa.text("status IN ('reviewing', 'pending')"),
)
@@ -0,0 +1,50 @@
"""新手引导视频播放记录表(领券浮层前 N 次替代广告)
见 app/models/guide_video.py:按账号计次(开播即计数)、play_token 幂等发币。
配置(开关 / 视频地址 / 次数 / 金币)复用既有 app_config 表,无需建表。
Revision ID: guide_video_play_table
Revises: meituan_coupon_feed_indexes
Create Date: 2026-07-23 12:00:00.000000
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "guide_video_play_table"
down_revision: Union[str, Sequence[str], None] = "meituan_coupon_feed_indexes"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.create_table(
"guide_video_play",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("user_id", sa.Integer(), nullable=False),
sa.Column("play_token", sa.String(length=64), nullable=False),
sa.Column("scene", sa.String(length=16), nullable=False, server_default="coupon"),
sa.Column("seq", sa.Integer(), nullable=False, server_default="1"),
sa.Column("video_url", sa.String(length=512), nullable=True),
sa.Column("coin", sa.Integer(), nullable=False, server_default="0"),
sa.Column("status", sa.String(length=16), nullable=False, server_default="playing"),
sa.Column("completed", sa.Integer(), nullable=False, server_default="0"),
sa.Column(
"started_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False
),
sa.Column("granted_at", sa.DateTime(timezone=True), nullable=True),
sa.ForeignKeyConstraint(["user_id"], ["user.id"]),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("play_token", name="uq_guide_video_play_token"),
)
op.create_index("ix_guide_video_play_user_id", "guide_video_play", ["user_id"])
op.create_index("ix_guide_video_play_started_at", "guide_video_play", ["started_at"])
def downgrade() -> None:
op.drop_index("ix_guide_video_play_started_at", table_name="guide_video_play")
op.drop_index("ix_guide_video_play_user_id", table_name="guide_video_play")
op.drop_table("guide_video_play")
@@ -0,0 +1,40 @@
"""guide_video_play 加 (user_id, seq) 唯一约束:堵住并发 /start 绕过次数上限
start_play 是无锁 check-then-insert(读 COUNT(*) 算 seq=used+1 再插一行),N 个并发
/start 会都读到同一个 used、算出同一个 seq、各插一行拿到各自的 play_token,于是 3 次
上限被绕过、每个 token 都能换 120 金币。加唯一键后并发同 seq 必撞,start_play 捕获
IntegrityError 降级为 should_play=false(客户端照旧放广告)。
用 unique index 而不是 batch_alter_table 加 UniqueConstraint:SQLite 加约束要整表重建,
而 CREATE UNIQUE INDEX 两边都原生支持,回滚也干净。
注:若库里已有并发产生的重复 (user_id, seq),建索引会失败 —— 本功能尚未上线,表通常是空的;
真撞上了先按 seq 去重(留 id 最小的一行,多发的金币按 scripts/reset_guide_video.py 的口径退)。
Revision ID: guide_video_user_seq_uq
Revises: d9c03cc3ea07
Create Date: 2026-07-24 10:00:00.000000
"""
from typing import Sequence, Union
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "guide_video_user_seq_uq"
down_revision: Union[str, Sequence[str], None] = "d9c03cc3ea07"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.create_index(
"uq_guide_video_play_user_seq",
"guide_video_play",
["user_id", "seq"],
unique=True,
)
def downgrade() -> None:
op.drop_index("uq_guide_video_play_user_seq", table_name="guide_video_play")
@@ -0,0 +1,52 @@
"""meituan_coupon 首页 feed 分页复合索引(销量最高 / 智能推荐)
「销量最高」「智能推荐」两个 tab 都是
WHERE city_id = ? [+ 过滤] → DISTINCT ON (dedup_key) ORDER BY dedup_key, <排序键> DESC
的形状。列顺序对齐后 Postgres 可以顺着索引流式去重,免掉「每翻一页就把该城全部券重排一遍」,
这是首页下滑到底越来越慢的根因之一(另一半在 app 层:见 api/v1/meituan.py 的 _paged_dedup_ids)。
⚠️ 本文件同时是一个 **merge 迁移**:主干此前有 3 个并行 head
(comparison_user_created_idx / monitoring_audit_rbac / notification_table),
`alembic upgrade head` 会因 multiple heads 报错。这里一并收敛回单 head。
Revision ID: meituan_coupon_feed_indexes
Revises: comparison_user_created_idx, monitoring_audit_rbac, notification_table
Create Date: 2026-07-23 10:00:00.000000
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "meituan_coupon_feed_indexes"
down_revision: Union[str, Sequence[str], None] = (
"comparison_user_created_idx",
"monitoring_audit_rbac",
"notification_table",
)
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# 销量最高:WHERE city_id=? AND sale_volume_num IS NOT NULL
# ORDER BY dedup_key, sale_volume_num DESC, commission_percent DESC
op.create_index(
"ix_meituan_coupon_city_dedup_sales",
"meituan_coupon",
["city_id", "dedup_key", sa.text("sale_volume_num DESC"), sa.text("commission_percent DESC")],
)
# 智能推荐:WHERE city_id=? AND commission_percent>=3.0
# ORDER BY dedup_key, commission_percent DESC
op.create_index(
"ix_meituan_coupon_city_dedup_comm",
"meituan_coupon",
["city_id", "dedup_key", sa.text("commission_percent DESC")],
)
def downgrade() -> None:
op.drop_index("ix_meituan_coupon_city_dedup_comm", table_name="meituan_coupon")
op.drop_index("ix_meituan_coupon_city_dedup_sales", table_name="meituan_coupon")
@@ -0,0 +1,210 @@
"""add generic behavior, risk incident and subject restriction tables
Revision ID: risk_monitor_generic
Revises: drop_withdraw_active_uniq_idx
"""
from __future__ import annotations
from collections.abc import Sequence
import sqlalchemy as sa
from sqlalchemy.dialects import postgresql
from alembic import op
revision: str = "risk_monitor_generic"
down_revision: str | None = "drop_withdraw_active_uniq_idx"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
_JSON = sa.JSON().with_variant(postgresql.JSONB(), "postgresql")
_RISK_MONITOR_PAGE = "risk-monitor"
_DEFAULT_RISK_MONITOR_ROLES = ("operator", "tech")
def _role_table() -> sa.TableClause:
return sa.table(
"admin_role",
sa.column("name", sa.String),
sa.column("pages", _JSON),
)
def _add_default_role_permissions() -> None:
"""Grant the new page without replacing any existing role customisation."""
role = _role_table()
conn = op.get_bind()
rows = conn.execute(
sa.select(role.c.name, role.c.pages).where(
role.c.name.in_(_DEFAULT_RISK_MONITOR_ROLES)
)
).all()
for name, pages in rows:
current_pages = list(pages or [])
if _RISK_MONITOR_PAGE not in current_pages:
conn.execute(
role.update()
.where(role.c.name == name)
.values(pages=[*current_pages, _RISK_MONITOR_PAGE])
)
def _remove_default_role_permissions() -> None:
role = _role_table()
conn = op.get_bind()
rows = conn.execute(
sa.select(role.c.name, role.c.pages).where(
role.c.name.in_(_DEFAULT_RISK_MONITOR_ROLES)
)
).all()
for name, pages in rows:
current_pages = list(pages or [])
if _RISK_MONITOR_PAGE in current_pages:
conn.execute(
role.update()
.where(role.c.name == name)
.values(
pages=[
page
for page in current_pages
if page != _RISK_MONITOR_PAGE
]
)
)
def upgrade() -> None:
op.create_table(
"behavior_event",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("event_type", sa.String(length=64), nullable=False),
sa.Column("subject_type", sa.String(length=32), nullable=False),
sa.Column("subject_id", sa.String(length=128), nullable=False),
sa.Column("user_id", sa.Integer(), nullable=True),
sa.Column("device_id", sa.String(length=128), nullable=True),
sa.Column("device_model", sa.String(length=128), nullable=True),
sa.Column("phone", sa.String(length=20), nullable=True),
sa.Column("client_ip", sa.String(length=64), nullable=True),
sa.Column("outcome", sa.String(length=24), nullable=False),
sa.Column("reason", sa.String(length=256), nullable=True),
sa.Column("details", sa.JSON(), nullable=True),
sa.Column("occurred_at", sa.DateTime(timezone=True), nullable=False),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.text("(CURRENT_TIMESTAMP)"),
nullable=False,
),
sa.PrimaryKeyConstraint("id"),
)
op.create_index("ix_behavior_event_occurred_at", "behavior_event", ["occurred_at"])
op.create_index(
"ix_behavior_event_type_time", "behavior_event", ["event_type", "occurred_at"]
)
op.create_index(
"ix_behavior_event_subject_time",
"behavior_event",
["subject_type", "subject_id", "occurred_at"],
)
op.create_index(
"ix_behavior_event_user_time", "behavior_event", ["user_id", "occurred_at"]
)
op.create_table(
"risk_incident",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("rule_code", sa.String(length=64), nullable=False),
sa.Column("event_type", sa.String(length=64), nullable=False),
sa.Column("subject_type", sa.String(length=32), nullable=False),
sa.Column("subject_id", sa.String(length=128), nullable=False),
sa.Column("window_key", sa.String(length=64), nullable=False),
sa.Column("window_start", sa.DateTime(timezone=True), nullable=False),
sa.Column("window_end", sa.DateTime(timezone=True), nullable=False),
sa.Column("first_event_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("triggered_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("last_event_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("event_count", sa.Integer(), nullable=False),
sa.Column("status", sa.String(length=24), nullable=False),
sa.Column("action_reason", sa.String(length=256), nullable=True),
sa.Column("handled_by", sa.Integer(), nullable=True),
sa.Column("handled_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("details", sa.JSON(), nullable=True),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.text("(CURRENT_TIMESTAMP)"),
nullable=False,
),
sa.Column(
"updated_at",
sa.DateTime(timezone=True),
server_default=sa.text("(CURRENT_TIMESTAMP)"),
nullable=False,
),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint(
"rule_code",
"subject_type",
"subject_id",
"window_key",
name="uq_risk_incident_rule_subject_window",
),
)
op.create_index(
"ix_risk_incident_rule_status",
"risk_incident",
["rule_code", "status", "triggered_at"],
)
op.create_index(
"ix_risk_incident_subject",
"risk_incident",
["subject_type", "subject_id", "triggered_at"],
)
op.create_table(
"subject_restriction",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("subject_type", sa.String(length=32), nullable=False),
sa.Column("subject_id", sa.String(length=128), nullable=False),
sa.Column("scope", sa.String(length=32), nullable=False),
sa.Column("active", sa.Boolean(), server_default=sa.true(), nullable=False),
sa.Column("reason", sa.String(length=256), nullable=True),
sa.Column("incident_id", sa.Integer(), nullable=True),
sa.Column("created_by", sa.Integer(), nullable=True),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.text("(CURRENT_TIMESTAMP)"),
nullable=False,
),
sa.Column("revoked_by", sa.Integer(), nullable=True),
sa.Column("revoked_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("details", sa.JSON(), nullable=True),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint(
"subject_type",
"subject_id",
"scope",
name="uq_subject_restriction_subject_scope",
),
)
op.create_index(
"ix_subject_restriction_lookup",
"subject_restriction",
["subject_type", "subject_id", "scope", "active"],
)
_add_default_role_permissions()
def downgrade() -> None:
_remove_default_role_permissions()
op.drop_index("ix_subject_restriction_lookup", table_name="subject_restriction")
op.drop_table("subject_restriction")
op.drop_index("ix_risk_incident_subject", table_name="risk_incident")
op.drop_index("ix_risk_incident_rule_status", table_name="risk_incident")
op.drop_table("risk_incident")
op.drop_index("ix_behavior_event_user_time", table_name="behavior_event")
op.drop_index("ix_behavior_event_subject_time", table_name="behavior_event")
op.drop_index("ix_behavior_event_type_time", table_name="behavior_event")
op.drop_index("ix_behavior_event_occurred_at", table_name="behavior_event")
op.drop_table("behavior_event")
@@ -0,0 +1,76 @@
"""add manual high-risk flag and note to user
Revision ID: user_manual_risk_fields
Revises: risk_monitor_generic
"""
from __future__ import annotations
from collections.abc import Sequence
import sqlalchemy as sa
from sqlalchemy.dialects import postgresql
from alembic import op
revision: str = "user_manual_risk_fields"
down_revision: str | None = "risk_monitor_generic"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
_JSON = sa.JSON().with_variant(postgresql.JSONB(), "postgresql")
_INVITE_WITHDRAW_PAGE = "invite-withdraws"
def _finance_role() -> sa.TableClause:
return sa.table(
"admin_role",
sa.column("name", sa.String),
sa.column("pages", _JSON),
)
def upgrade() -> None:
with op.batch_alter_table("user") as batch_op:
batch_op.add_column(
sa.Column(
"is_high_risk",
sa.Boolean(),
server_default=sa.false(),
nullable=False,
)
)
batch_op.add_column(sa.Column("high_risk_note", sa.Text(), nullable=True))
batch_op.create_index("ix_user_is_high_risk", ["is_high_risk"], unique=False)
role = _finance_role()
conn = op.get_bind()
row = conn.execute(
sa.select(role.c.pages).where(role.c.name == "finance")
).scalar_one_or_none()
if row is not None and _INVITE_WITHDRAW_PAGE not in (row or []):
conn.execute(
role.update()
.where(role.c.name == "finance")
.values(pages=[*(row or []), _INVITE_WITHDRAW_PAGE])
)
def downgrade() -> None:
role = _finance_role()
conn = op.get_bind()
row = conn.execute(
sa.select(role.c.pages).where(role.c.name == "finance")
).scalar_one_or_none()
if row is not None:
conn.execute(
role.update()
.where(role.c.name == "finance")
.values(
pages=[
page for page in (row or []) if page != _INVITE_WITHDRAW_PAGE
]
)
)
with op.batch_alter_table("user") as batch_op:
batch_op.drop_index("ix_user_is_high_risk")
batch_op.drop_column("high_risk_note")
batch_op.drop_column("is_high_risk")
+9 -2
View File
@@ -17,6 +17,7 @@ from app.admin.routers.ad_audit import router as ad_audit_router
from app.admin.routers.ad_config import router as ad_config_router
from app.admin.routers.ad_revenue import router as ad_revenue_router
from app.admin.routers.admins import router as admins_router
from app.admin.routers.analytics_health import router as analytics_health_router
from app.admin.routers.audit import router as audit_router
from app.admin.routers.auth import router as auth_router
from app.admin.routers.comparison import router as comparison_router
@@ -25,21 +26,23 @@ from app.admin.routers.coupon_data import router as coupon_data_router
from app.admin.routers.cps import router as cps_router
from app.admin.routers.dashboard import router as dashboard_router
from app.admin.routers.device_liveness import router as device_liveness_router
from app.admin.routers.ops_stat_config import router as ops_stat_config_router
from app.admin.routers.analytics_health import router as analytics_health_router
from app.admin.routers.event_logs import router as event_logs_router
from app.admin.routers.feedback import router as feedback_router
from app.admin.routers.feedback_qr import router as feedback_qr_router
from app.admin.routers.guide_video import router as guide_video_router
from app.admin.routers.huawei_review import router as huawei_review_router
from app.admin.routers.onboarding import router as onboarding_router
from app.admin.routers.ops_marquee_seed import router as ops_marquee_seed_router
from app.admin.routers.ops_stat_config import router as ops_stat_config_router
from app.admin.routers.price_report import router as price_report_router
from app.admin.routers.risk_monitor import router as risk_monitor_router
from app.admin.routers.roles import router as roles_router
from app.admin.routers.users import router as users_router
from app.admin.routers.wallet import router as wallet_router
from app.admin.routers.withdraw import router as withdraw_router
from app.core.config import settings
from app.core.logging import setup_logging
from app.integrations import meituan as mt_meituan
setup_logging(debug=settings.APP_DEBUG)
logger = logging.getLogger("shagua.admin")
@@ -53,6 +56,8 @@ async def lifespan(_: FastAPI) -> AsyncIterator[None]:
settings.DATABASE_URL.split("://", 1)[0],
)
yield
# CPS 后台页会打美团(routers/cps.py),那条共享 client 若被建过要在这里关掉连接池
mt_meituan.close_client()
logger.info("admin app shutting down")
@@ -97,10 +102,12 @@ admin_app.include_router(onboarding_router)
admin_app.include_router(wallet_router)
admin_app.include_router(withdraw_router)
admin_app.include_router(price_report_router)
admin_app.include_router(risk_monitor_router)
admin_app.include_router(feedback_router)
admin_app.include_router(event_logs_router)
admin_app.include_router(analytics_health_router)
admin_app.include_router(feedback_qr_router)
admin_app.include_router(guide_video_router)
admin_app.include_router(admins_router)
admin_app.include_router(roles_router)
admin_app.include_router(audit_router)
+6 -4
View File
@@ -23,7 +23,8 @@ PERMISSION_CATALOG: list[dict] = [
{"key": "cps", "label": "CPS收益"},
]},
{"group": "奖励审核", "pages": [
{"key": "withdraws", "label": "提现审核"},
{"key": "invite-withdraws", "label": "邀请提现审核"},
{"key": "withdraws", "label": "其他提现审核"},
{"key": "price-reports", "label": "低价审核"},
{"key": "feedbacks", "label": "用户反馈"},
]},
@@ -34,6 +35,7 @@ PERMISSION_CATALOG: list[dict] = [
{"key": "users", "label": "用户管理"},
]},
{"group": "监控审计", "pages": [
{"key": "risk-monitor", "label": "风控监控"},
{"key": "device-liveness", "label": "设备存活"},
{"key": "analytics-health", "label": "埋点成功率"},
{"key": "event-logs", "label": "埋点日志"},
@@ -55,13 +57,13 @@ BUILTIN_ROLES: list[dict] = [
{"name": SUPER_ADMIN_ROLE, "label": "管理员", "pages": []},
{"name": "operator", "label": "运营", "pages": [
"dashboard", "coupon-data", "ad-revenue-report", "comparison-records",
"cps", "device-liveness", "price-reports", "feedbacks", "huawei-review",
"cps", "risk-monitor", "device-liveness", "price-reports", "feedbacks", "huawei-review",
]},
{"name": "finance", "label": "财务", "pages": [
"dashboard", "ad-revenue-report", "cps", "withdraws",
"dashboard", "ad-revenue-report", "cps", "invite-withdraws", "withdraws",
]},
{"name": "tech", "label": "技术", "pages": [
"dashboard", "device-liveness", "analytics-health", "config", "ad-revenue", "huawei-review",
"dashboard", "risk-monitor", "device-liveness", "analytics-health", "config", "ad-revenue", "huawei-review",
"event-logs", "audit-logs",
]},
]
+101 -3
View File
@@ -12,10 +12,11 @@
"""
from __future__ import annotations
from sqlalchemy import func, select
from sqlalchemy import func, or_, select
from sqlalchemy.orm import Session
from app.core import rewards
from app.models.ad_ecpm import AdEcpmRecord
from app.models.ad_feed_reward import AdFeedRewardRecord
from app.models.ad_reward import AdRewardRecord
from app.repositories.ad_feed_reward import FEED_REWARD_UNIT_SECONDS
@@ -55,10 +56,21 @@ def _reward_video_rows(
if user_id is not None:
stmt = stmt.where(AdRewardRecord.user_id == user_id)
records = list(db.execute(stmt).scalars())
# S2S 发奖回调不携带实际填充 ADN;用相同用户和 ad_session_id 的展示记录回填。
session_ids = {record.ad_session_id for record in records if record.ad_session_id}
impression_by_session = {
(record.user_id, record.ad_session_id): record
for record in db.execute(
select(AdEcpmRecord).where(AdEcpmRecord.ad_session_id.in_(session_ids))
).scalars()
} if session_ids else {}
# 用本日之前的累计份数做起点,当日 granted 在其上继续递增 → 与 _granted_cumulative+1 对齐
granted_n: dict[int, int] = _prior_granted_counts(db, date=date, user_id=user_id)
rows: list[dict] = []
for rec in db.execute(stmt).scalars():
for rec in records:
impression = impression_by_session.get((rec.user_id, rec.ad_session_id))
if rec.status == "granted":
nth = granted_n.get(rec.user_id, 0) + 1
granted_n[rec.user_id] = nth
@@ -68,6 +80,8 @@ def _reward_video_rows(
"record_id": rec.id,
"user_id": rec.user_id,
"ad_session_id": rec.ad_session_id,
"adn": impression.adn if impression is not None else None,
"slot_id": impression.slot_id if impression is not None else None,
"app_env": rec.app_env,
"our_code_id": rec.our_code_id,
"created_at": rec.created_at,
@@ -90,6 +104,8 @@ def _reward_video_rows(
"record_id": rec.id,
"user_id": rec.user_id,
"ad_session_id": rec.ad_session_id,
"adn": impression.adn if impression is not None else None,
"slot_id": impression.slot_id if impression is not None else None,
"app_env": rec.app_env,
"our_code_id": rec.our_code_id,
"created_at": rec.created_at,
@@ -149,6 +165,78 @@ def _feed_scene_matches(rec: AdFeedRewardRecord, scene: str | None) -> bool:
return True
def _nonblank(value: str | None) -> str | None:
value = value.strip() if value else None
return value or None
def _unique_ad_source(records: list[AdEcpmRecord]) -> tuple[str | None, str | None]:
"""仅在候选展示记录指向唯一 ADN 时回填来源,避免错误归因。"""
adns = {_nonblank(record.adn) for record in records}
adns.discard(None)
if len(adns) != 1:
return None, None
slots = {_nonblank(record.slot_id) for record in records}
slots.discard(None)
return next(iter(adns)), next(iter(slots)) if len(slots) == 1 else None
def _feed_source_fallbacks(
db: Session, records: list[AdFeedRewardRecord]
) -> tuple[
dict[tuple[int, str], tuple[str | None, str | None]],
dict[tuple[int, str, str], tuple[str | None, str | None]],
]:
"""为旧信息流发奖记录构建安全来源索引。"""
session_ids = {record.ad_session_id for record in records if record.ad_session_id}
trace_ids = {record.trace_id for record in records if record.trace_id}
if not session_ids and not trace_ids:
return {}, {}
filters = []
if session_ids:
filters.append(AdEcpmRecord.ad_session_id.in_(session_ids))
if trace_ids:
filters.append(AdEcpmRecord.trace_id.in_(trace_ids))
impressions = list(db.execute(select(AdEcpmRecord).where(or_(*filters))).scalars())
by_session: dict[tuple[int, str], list[AdEcpmRecord]] = {}
by_trace_ecpm: dict[tuple[int, str, str], list[AdEcpmRecord]] = {}
for impression in impressions:
if impression.ad_session_id:
by_session.setdefault((impression.user_id, impression.ad_session_id), []).append(impression)
if impression.trace_id:
by_trace_ecpm.setdefault(
(impression.user_id, impression.trace_id, impression.ecpm_raw), []
).append(impression)
return (
{key: _unique_ad_source(value) for key, value in by_session.items()},
{key: _unique_ad_source(value) for key, value in by_trace_ecpm.items()},
)
def _feed_source(
record: AdFeedRewardRecord,
*,
by_session: dict[tuple[int, str], tuple[str | None, str | None]],
by_trace_ecpm: dict[tuple[int, str, str], tuple[str | None, str | None]],
) -> tuple[str | None, str | None]:
"""返回本条发奖广告的来源;无唯一证据时保留原始空值。"""
adn, slot_id = _nonblank(record.adn), _nonblank(record.slot_id)
if adn and slot_id:
return adn, slot_id
candidate = by_session.get((record.user_id, record.ad_session_id or ""))
if candidate is None and record.trace_id:
candidate = by_trace_ecpm.get((record.user_id, record.trace_id, record.ecpm_raw))
if candidate is None:
return adn, slot_id
candidate_adn, candidate_slot_id = candidate
return adn or candidate_adn, slot_id or candidate_slot_id
def _feed_rows(
db: Session, *, date: str, user_id: int | None, scene: str | None = None
) -> list[dict]:
@@ -167,11 +255,17 @@ def _feed_rows(
if user_id is not None:
stmt = stmt.where(AdFeedRewardRecord.user_id == user_id)
records = list(db.execute(stmt).scalars())
by_session, by_trace_ecpm = _feed_source_fallbacks(db, records)
# 本日之前的累计**条数**做起点,与发奖侧 granted_unit_total(COUNT granted)对齐
granted_count: dict[int, int] = _feed_prior_granted_count(db, date=date, user_id=user_id)
rows: list[dict] = []
for rec in db.execute(stmt).scalars():
for rec in records:
keep = _feed_scene_matches(rec, scene) # 累计照常推进,这里只决定是否展示本行
adn, slot_id = _feed_source(
rec, by_session=by_session, by_trace_ecpm=by_trace_ecpm
)
if rec.status == "granted":
# 一条广告 = 1 份(与 grant_feed_reward 同口径:看满一份即发该条满额,不按 unit_count 累加)。
# nth = 账号累计第几**条**(含本日之前),与发奖侧 granted_unit_total+1 对齐;累计照常推进
@@ -188,6 +282,8 @@ def _feed_rows(
"record_id": rec.id,
"user_id": rec.user_id,
"ad_session_id": rec.ad_session_id,
"adn": adn,
"slot_id": slot_id,
"trace_id": rec.trace_id,
"app_env": rec.app_env,
"our_code_id": rec.our_code_id,
@@ -214,6 +310,8 @@ def _feed_rows(
"record_id": rec.id,
"user_id": rec.user_id,
"ad_session_id": rec.ad_session_id,
"adn": adn,
"slot_id": slot_id,
"trace_id": rec.trace_id,
"app_env": rec.app_env,
"our_code_id": rec.our_code_id,
+45 -8
View File
@@ -43,7 +43,7 @@ _KNOWN_PROD_BUSINESS_CODE_IDS = frozenset({"104098712", "104099389"})
_TEST_BUSINESS_CODE_IDS = frozenset({"104127529", "104127626", "104137445"})
def _business_code_ids(db: Session, app_env: str | None) -> set[str]:
def business_code_ids(db: Session, app_env: str | None) -> set[str]:
"""返回指定应用环境下可用于业务收益对账的 GroMore 聚合代码位。"""
prod_config = app_config.get_ad_config(db)
prod_ids = set(_KNOWN_PROD_BUSINESS_CODE_IDS) | {
@@ -96,7 +96,11 @@ _REWARD_DETAIL_KEYS = (
def _reward_detail(row: dict) -> dict:
"""从 audit 行抽出发奖复算明细(给前端展开行渲染因子1/因子2/份数/LT/应发实发)。"""
return {k: row[k] for k in _REWARD_DETAIL_KEYS}
detail = {key: row[key] for key in _REWARD_DETAIL_KEYS}
# 聚合父行可能包含多个 ADN,来源必须保留在每一条发奖明细上。
detail["adn"] = row.get("adn")
detail["slot_id"] = row.get("slot_id")
return detail
def ad_revenue_report(
@@ -244,8 +248,8 @@ def ad_revenue_report(
"impressions": 0,
"ecpm": row["ecpm"],
"revenue_yuan": 0.0,
"adn": None,
"slot_id": None,
"adn": row.get("adn"),
"slot_id": row.get("slot_id"),
"has_reward": True,
"status": row["status"],
"expected_coin": int(row["expected_coin"]),
@@ -316,10 +320,10 @@ def ad_revenue_report(
# 业务口径仅保留正式配置/测试业务链路实际使用的代码位。穿山甲“全量”还包含广告测试
# demo、插屏等没有客户端收益上报的曝光,两边直接比较会天然产生假差额。
business_code_ids: set[str] | None = None
business_ids: set[str] | None = None
if revenue_scope == "business":
business_code_ids = _business_code_ids(db, app_env)
events = [e for e in events if e.get("our_code_id") in business_code_ids]
business_ids = business_code_ids(db, app_env)
events = [e for e in events if e.get("our_code_id") in business_ids]
# 排序:time=按时间倒序(新→旧);ecpm=按 eCPM 数值倒序(eCPM 原值是字符串「分」,转数值排;
# 纯发奖行用其发奖采用的 eCPM,缺失/非法计 0 排末尾)。
@@ -377,7 +381,7 @@ def ad_revenue_report(
date_from=date_from,
date_to=date_to,
app_env=app_env,
our_code_ids=business_code_ids,
our_code_ids=business_ids,
)
if pangle_aggs:
by_date = {a["date"]: a for a in pangle_aggs}
@@ -427,6 +431,38 @@ def ad_revenue_report(
for k, v in type_map.items()
}
# 经营看板的两类 eCPM 必须按真实展示的 SDK eCPM 加权,不能用发奖状态修正后的
# 收益反推。Draw 包含新 draw 与历史 feed;看视频包含福利与提现视频。
category_map: dict[str, dict] = {}
for event in events:
category = (
"draw" if event["ad_type"] in {"draw", "feed"}
else "video" if event["ad_type"] in {"reward_video", "withdrawal_video"}
else None
)
if category is None:
continue
stat = category_map.setdefault(category, {
"impressions": 0,
"revenue_yuan": 0.0,
"ecpm_fen_sum": 0.0,
})
impressions = int(event["impressions"])
stat["impressions"] += impressions
stat["revenue_yuan"] += event["revenue_yuan"]
stat["ecpm_fen_sum"] += rewards.parse_ecpm_fen(event["ecpm"]) * impressions
category_stats = {
key: {
"impressions": value["impressions"],
"revenue_yuan": round(value["revenue_yuan"], 6),
"ecpm_yuan": round(
value["ecpm_fen_sum"] / value["impressions"] / 100.0,
6,
) if value["impressions"] else 0.0,
}
for key, value in category_map.items()
}
# 分场景小计(按 feed_scene:展示条数 + 预估收益),同 type_stats 基于全量 events——
# 供数据大盘「领券广告 / 比价广告」卡用。此前大盘是在分页 items 里按 feed_scene 现算,
# 2026-07-02 起信息流逐条展示行(唯一带收益 + 场景的行)不再进主表 items,现算恒为 0;
@@ -481,6 +517,7 @@ def ad_revenue_report(
"daily": daily,
"hourly": hourly,
"type_stats": type_stats,
"category_stats": category_stats,
"scene_stats": scene_stats,
"dau": dau,
"items": main_rows[offset:offset + limit],
+12 -12
View File
@@ -16,7 +16,7 @@ from sqlalchemy import case, func, or_, select
from sqlalchemy.orm import Session
from app.core import rewards
from app.models.coupon_state import CouponClaimRecord, CouponSession
from app.models.coupon_state import CouponClaimEvent, CouponClaimRecord, CouponSession
from app.models.user import User
from app.repositories import ad_ecpm as crud_ecpm
from app.repositories.coupon_state import DEFAULT_PLATFORMS, coupon_id_to_platform
@@ -193,18 +193,18 @@ def _point_scores_by_trace(db: Session, trace_ids: list[str]) -> dict[str, dict[
"""聚合查询批量返回逐场点位分数,不加载逐券明细。"""
if not trace_ids:
return {}
succeeded = func.sum(case((CouponClaimRecord.status.in_(_SLOT_OK), 1), else_=0))
succeeded = func.sum(case((CouponClaimEvent.status.in_(_SLOT_OK), 1), else_=0))
rows = db.execute(
select(
CouponClaimRecord.trace_id,
CouponClaimEvent.trace_id,
succeeded.label("succeeded"),
func.count().label("tried"),
)
.where(
CouponClaimRecord.trace_id.in_(trace_ids),
CouponClaimRecord.status.in_(_SLOT_TRIED),
CouponClaimEvent.trace_id.in_(trace_ids),
CouponClaimEvent.status.in_(_SLOT_TRIED),
)
.group_by(CouponClaimRecord.trace_id)
.group_by(CouponClaimEvent.trace_id)
).all()
return {
trace_id: {"succeeded": int(success_count or 0), "tried": int(tried or 0)}
@@ -217,13 +217,13 @@ def coupon_point_details(db: Session, *, trace_id: str) -> list[dict]:
"""按单个 trace 查询逐券结果;仅在后台用户点击分数时调用。"""
rows = db.execute(
select(
CouponClaimRecord.coupon_id,
CouponClaimRecord.coupon_name,
CouponClaimRecord.status,
CouponClaimRecord.reason,
CouponClaimEvent.coupon_id,
CouponClaimEvent.coupon_name,
CouponClaimEvent.status,
CouponClaimEvent.reason,
)
.where(CouponClaimRecord.trace_id == trace_id)
.order_by(CouponClaimRecord.id)
.where(CouponClaimEvent.trace_id == trace_id)
.order_by(CouponClaimEvent.id)
).all()
return [
{
+19
View File
@@ -44,6 +44,25 @@ def set_user_debug_trace(
return user
def set_user_risk(
db: Session,
user: User,
*,
is_high_risk: bool,
note: str | None,
commit: bool = True,
) -> User:
"""设置人工风险结论和备注,支持与审计日志共用同一事务。"""
user.is_high_risk = is_high_risk
user.high_risk_note = note.strip() if is_high_risk and note else None
if commit:
db.commit()
db.refresh(user)
else:
db.flush()
return user
def update_feedback_status(
db: Session, feedback: Feedback, *, status: str, commit: bool = True
) -> Feedback:
+385 -194
View File
@@ -9,11 +9,12 @@ from datetime import date, datetime, time, timedelta, timezone
from decimal import ROUND_HALF_UP, Decimal
from zoneinfo import ZoneInfo
from sqlalchemy import Select, asc, case, desc, func, or_, select
from sqlalchemy import Select, and_, asc, case, desc, func, or_, select
from sqlalchemy.orm import Session
from app.core import rewards
from app.core.config import settings
from app.models.ad_ecpm import AdEcpmRecord
from app.models.ad_feed_reward import AdFeedRewardRecord
from app.models.ad_reward import AdRewardRecord
from app.models.admin import AdminAuditLog
@@ -22,14 +23,15 @@ from app.models.comparison import ComparisonRecord
from app.models.coupon_state import CouponPromptEngagement
from app.models.device import DeviceLiveness
from app.models.feedback import Feedback
from app.models.invite import InviteRelation
from app.models.onboarding import OnboardingCompletion
from app.models.price_report import PriceReport
from app.models.savings import SavingsRecord
from app.models.user import User
from app.models.wallet import (
CashTransaction,
CoinAccount,
CoinTransaction,
InviteCashTransaction,
WithdrawOrder,
)
from app.repositories import activity, ad_ecpm
@@ -45,6 +47,106 @@ _FEED_SCENE_LABEL = {
}
_DEVICE_MARKETING_NAMES = {
"23078RKD5C": "Redmi K60 至尊版",
"M2012K11AC": "Redmi K40",
"PJA110": "一加 Ace 2 Pro",
"PPG-AN00": "荣耀 GT Pro",
"V2166BA": "vivo Y77e",
"V2309A": "vivo X100",
}
def _device_marketing_name(model: str | None) -> str | None:
"""把线上已知 Build.MODEL 编码转成用户可识别的商品名。"""
if not model:
return None
return _DEVICE_MARKETING_NAMES.get(model.strip().upper())
def _attach_comparison_order_status(db: Session, items: list[ComparisonRecord]) -> None:
"""按 C 端既有口径给比价记录批量补充是否真实下单。"""
user_ids = {item.user_id for item in items if item.user_id is not None}
shop_names = {item.store_name for item in items if item.store_name}
ordered_pairs: set[tuple[int, str]] = set()
if user_ids and shop_names:
rows = db.execute(
select(SavingsRecord.user_id, SavingsRecord.shop_name)
.where(
SavingsRecord.user_id.in_(user_ids),
SavingsRecord.source == "compare",
SavingsRecord.shop_name.in_(shop_names),
)
.distinct()
).all()
ordered_pairs = {
(row.user_id, row.shop_name)
for row in rows
if row.shop_name is not None
}
for item in items:
item.ordered = bool(
item.user_id is not None
and item.store_name
and (item.user_id, item.store_name) in ordered_pairs
)
def _attach_comparison_device_details(items: list[ComparisonRecord]) -> None:
"""给比价记录补充可读机型名,同时保留原始设备编码。"""
for item in items:
item.device_model_name = _device_marketing_name(item.device_model)
def _attach_feedback_device_details(db: Session, feedbacks: list[Feedback]) -> None:
"""按同一用户、同一设备编码及提交时间补齐厂商和 ROM 大版本。"""
candidates = [
item for item in feedbacks if item.device_model and item.device_model.strip()
]
for item in candidates:
item.device_model_name = _device_marketing_name(item.device_model)
item.device_manufacturer = None
item.rom_version = None
if not candidates:
return
ranked = (
select(
Feedback.id.label("feedback_id"),
ComparisonRecord.device_manufacturer.label("device_manufacturer"),
ComparisonRecord.rom_version.label("rom_version"),
func.row_number()
.over(
partition_by=Feedback.id,
order_by=(
ComparisonRecord.created_at.desc(),
ComparisonRecord.id.desc(),
),
)
.label("row_num"),
)
.join(
ComparisonRecord,
and_(
ComparisonRecord.user_id == Feedback.user_id,
ComparisonRecord.device_model == Feedback.device_model,
ComparisonRecord.created_at <= Feedback.created_at,
),
)
.where(Feedback.id.in_([item.id for item in candidates]))
.subquery()
)
details = {
row.feedback_id: row
for row in db.execute(select(ranked).where(ranked.c.row_num == 1)).all()
}
for item in candidates:
detail = details.get(item.id)
item.device_manufacturer = detail.device_manufacturer if detail else None
item.rom_version = detail.rom_version if detail else None
def cursor_paginate(
db: Session, stmt: Select, id_col, *, limit: int, cursor: int | None
) -> tuple[list, int | None]:
@@ -277,6 +379,8 @@ def list_comparison_records(
limit=limit, cursor=cursor,
)
_attach_user_info(db, items)
_attach_comparison_order_status(db, items)
_attach_comparison_device_details(items)
# 「本次比价看广告的预估收益」:按本页 trace_id 一次性聚合(同 _attach_user_info 逐页范式)。
# ad_revenue_yuan 非 ORM 列,仅瞬态挂实例上供 AdminComparisonListItem(from_attributes)读出。
rev = ad_ecpm.revenue_yuan_by_trace(db, [it.trace_id for it in items])
@@ -423,6 +527,8 @@ def get_comparison_record(db: Session, record_id: int) -> ComparisonRecord | Non
rec = db.get(ComparisonRecord, record_id)
if rec is not None:
_attach_user_info(db, [rec])
_attach_comparison_order_status(db, [rec])
_attach_comparison_device_details([rec])
return rec
@@ -710,16 +816,7 @@ def list_all_withdraw_orders(
elif quick_filter == "today":
stmt = stmt.where(WithdrawOrder.created_at >= today_start)
elif quick_filter == "high_risk":
stmt = stmt.where(
or_(
WithdrawOrder.user_name.is_(None),
WithdrawOrder.user_name == "",
User.status != "active",
User.created_at >= now - timedelta(hours=24),
WithdrawOrder.status.in_(("failed", "rejected")),
WithdrawOrder.fail_reason.is_not(None),
)
)
stmt = stmt.where(User.is_high_risk.is_(True))
sort_cols = {
"id": WithdrawOrder.id,
@@ -745,7 +842,9 @@ def _as_utc(value: datetime) -> datetime:
return value.astimezone(timezone.utc)
def withdraw_list_enrichment(db: Session, user_ids: list[int]) -> dict[int, dict]:
def withdraw_list_enrichment(
db: Session, user_ids: list[int], *, source: str | None = None
) -> dict[int, dict]:
"""批量富化提现单列表:按本页 user_id 取 手机号/昵称 + 各自累计成功提现金额(分)。
两条聚合查询搞定(避免逐行 N+1)。累计口径与 get_user_overview 的 withdraw_success_cents
@@ -755,25 +854,32 @@ def withdraw_list_enrichment(db: Session, user_ids: list[int]) -> dict[int, dict
uniq = list(set(user_ids))
if not uniq:
return {}
success_rows = db.execute(
select(
WithdrawOrder.user_id,
func.coalesce(func.sum(WithdrawOrder.amount_cents), 0),
)
.where(WithdrawOrder.user_id.in_(uniq), WithdrawOrder.status == "success")
.group_by(WithdrawOrder.user_id)
).all()
success_stmt = select(
WithdrawOrder.user_id,
func.coalesce(func.sum(WithdrawOrder.amount_cents), 0),
).where(WithdrawOrder.user_id.in_(uniq), WithdrawOrder.status == "success")
if source:
success_stmt = success_stmt.where(WithdrawOrder.source == source)
success_rows = db.execute(success_stmt.group_by(WithdrawOrder.user_id)).all()
success_map = {uid: int(total) for uid, total in success_rows}
users = db.execute(
select(User.id, User.phone, User.nickname).where(User.id.in_(uniq))
select(
User.id,
User.phone,
User.nickname,
User.is_high_risk,
User.high_risk_note,
).where(User.id.in_(uniq))
).all()
return {
uid: {
"phone": phone,
"nickname": nickname,
"is_high_risk": is_high_risk,
"high_risk_note": high_risk_note,
"cumulative_success_cents": success_map.get(uid, 0),
}
for uid, phone, nickname in users
for uid, phone, nickname, is_high_risk, high_risk_note in users
}
@@ -817,6 +923,7 @@ def list_feedbacks(
db, stmt, (order_fn(sort_col), id_order), limit=limit, cursor=cursor
)
_attach_user_info(db, items) # 列表展示完整手机号(点手机号查该用户全部反馈)
_attach_feedback_device_details(db, items)
return items, next_cursor, total
@@ -879,15 +986,16 @@ def get_withdraw_by_out_bill_no(db: Session, out_bill_no: str) -> WithdrawOrder
).scalar_one_or_none()
def withdraw_summary(db: Session) -> dict:
"""提现审核台顶部统计。金额单位:分。"""
rows = db.execute(
select(
WithdrawOrder.status,
func.count(WithdrawOrder.id),
func.coalesce(func.sum(WithdrawOrder.amount_cents), 0),
).group_by(WithdrawOrder.status)
).all()
def withdraw_summary(db: Session, *, source: str | None = None) -> dict:
"""提现审核台顶部统计。金额单位:分;口径与列表的 source 筛选一致"""
summary_stmt = select(
WithdrawOrder.status,
func.count(WithdrawOrder.id),
func.coalesce(func.sum(WithdrawOrder.amount_cents), 0),
)
if source:
summary_stmt = summary_stmt.where(WithdrawOrder.source == source)
rows = db.execute(summary_stmt.group_by(WithdrawOrder.status)).all()
by_status = {
status: {"count": int(count), "amount_cents": int(amount_cents)}
for status, count, amount_cents in rows
@@ -900,31 +1008,154 @@ def withdraw_summary(db: Session) -> dict:
)
def _today_count(status: str) -> int:
return db.execute(
select(func.count(WithdrawOrder.id)).where(
WithdrawOrder.status == status,
WithdrawOrder.updated_at >= today_start,
)
).scalar_one()
today_success_amount = db.execute(
select(func.coalesce(func.sum(WithdrawOrder.amount_cents), 0)).where(
WithdrawOrder.status == "success",
stmt = select(func.count(WithdrawOrder.id)).where(
WithdrawOrder.status == status,
WithdrawOrder.updated_at >= today_start,
)
).scalar_one()
if source:
stmt = stmt.where(WithdrawOrder.source == source)
return db.execute(stmt).scalar_one()
today_success_amount_stmt = select(
func.coalesce(func.sum(WithdrawOrder.amount_cents), 0)
).where(
WithdrawOrder.status == "success",
WithdrawOrder.updated_at >= today_start,
)
if source:
today_success_amount_stmt = today_success_amount_stmt.where(
WithdrawOrder.source == source
)
today_success_amount = db.execute(today_success_amount_stmt).scalar_one()
def _status_count(status: str) -> int:
return by_status.get(status, {}).get("count", 0)
return {
"reviewing_count": by_status.get("reviewing", {}).get("count", 0),
"reviewing_count": _status_count("reviewing"),
"reviewing_amount_cents": by_status.get("reviewing", {}).get("amount_cents", 0),
"pending_count": by_status.get("pending", {}).get("count", 0),
"failed_count": by_status.get("failed", {}).get("count", 0),
"pending_count": _status_count("pending"),
"success_count": _status_count("success"),
"rejected_count": _status_count("rejected"),
"failed_count": _status_count("failed"),
"total_count": sum(item["count"] for item in by_status.values()),
"today_success_count": _today_count("success"),
"today_success_amount_cents": int(today_success_amount),
"today_rejected_count": _today_count("rejected"),
}
def invite_overview(
db: Session,
inviter_user_id: int,
*,
date_from: datetime | None = None,
date_to: datetime | None = None,
) -> dict:
"""邀请提现详情:邀请关系统计及受邀用户首次比价/首单信息。
首次记录按自增 id 取最早一条,批量查询避免按受邀用户逐行查询。
邀请成功口径为完成首次比价并已发邀请奖励(compare_reward_granted)。
"""
relation_stmt = (
select(InviteRelation, User)
.join(User, User.id == InviteRelation.invitee_user_id)
.where(InviteRelation.inviter_user_id == inviter_user_id)
.order_by(InviteRelation.created_at.asc(), InviteRelation.id.asc())
)
if date_from is not None:
relation_stmt = relation_stmt.where(User.created_at >= _as_utc_naive(date_from))
if date_to is not None:
relation_stmt = relation_stmt.where(User.created_at <= _as_utc_naive(date_to))
relation_rows = db.execute(relation_stmt).all()
invitee_ids = [relation.invitee_user_id for relation, _ in relation_rows]
if not invitee_ids:
return {"invite_total": 0, "invite_success_total": 0, "items": []}
first_compare_ids = list(
db.execute(
select(func.min(ComparisonRecord.id))
.where(
ComparisonRecord.user_id.in_(invitee_ids),
ComparisonRecord.status == "success",
)
.group_by(ComparisonRecord.user_id)
).scalars()
)
comparisons = (
db.execute(
select(
ComparisonRecord.user_id,
ComparisonRecord.store_name,
ComparisonRecord.product_names,
).where(ComparisonRecord.id.in_(first_compare_ids))
).all()
if first_compare_ids
else []
)
comparison_map = {
user_id: (store_name, product_names)
for user_id, store_name, product_names in comparisons
}
first_order_ids = list(
db.execute(
select(func.min(SavingsRecord.id))
.where(
SavingsRecord.user_id.in_(invitee_ids),
SavingsRecord.source == "compare",
)
.group_by(SavingsRecord.user_id)
).scalars()
)
orders = (
db.execute(
select(
SavingsRecord.user_id,
SavingsRecord.shop_name,
SavingsRecord.title,
SavingsRecord.dishes,
SavingsRecord.order_amount_cents,
).where(SavingsRecord.id.in_(first_order_ids))
).all()
if first_order_ids
else []
)
order_map = {
user_id: (
shop_name,
title or "".join(dishes or []),
order_amount_cents,
)
for user_id, shop_name, title, dishes, order_amount_cents in orders
}
items = []
for relation, user in relation_rows:
compare_store, compare_products = comparison_map.get(user.id, (None, None))
order_store, order_products, order_amount = order_map.get(
user.id, (None, None, None)
)
items.append(
{
"user_id": user.id,
"phone": user.phone,
"registered_at": user.created_at,
"invite_success": bool(relation.compare_reward_granted),
"first_compare_store": compare_store,
"first_compare_products": compare_products,
"first_order_store": order_store,
"first_order_products": order_products,
"first_order_amount_cents": order_amount,
}
)
return {
"invite_total": len(items),
"invite_success_total": sum(1 for item in items if item["invite_success"]),
"items": items,
}
def list_withdraw_audit_logs(
db: Session, out_bill_no: str, *, limit: int = 20
) -> list[AdminAuditLog]:
@@ -945,8 +1176,6 @@ def withdraw_risk_flags(
cash_balance_cents: int,
) -> tuple[list[str], int]:
flags: list[str] = []
if not order.user_name:
flags.append("缺少提现实名")
if user and user.status != "active":
flags.append(f"账号状态:{user.status}")
if user and user.created_at:
@@ -969,125 +1198,6 @@ def withdraw_risk_flags(
return flags, score
def _check_withdraw_ledger_side(
orders: list[WithdrawOrder], txns: list, *, withdraw_biz: str, refund_biz: str
) -> dict:
"""对某一本账(普通现金 / 邀请奖励金)做提现单 ↔ 流水的交叉校验。
orders 已按 source 过滤到本账;txns 是本账流水表里 withdraw_biz/refund_biz 两类流水。
规则:每单发起应有一条扣款流水(ref_id=out_bill_no);失败/拒绝单应有且仅一条退款流水;
非退款终态不应出现退款流水。四个计数全为 0 即本账自洽。
"""
withdraw_refs = {txn.ref_id for txn in txns if txn.biz_type == withdraw_biz}
refund_counts: dict[str, int] = {}
for txn in txns:
if txn.biz_type == refund_biz and txn.ref_id:
refund_counts[txn.ref_id] = refund_counts.get(txn.ref_id, 0) + 1
missing_withdraw = 0
missing_refund = 0
refund_on_non_terminal = 0
for order in orders:
if order.out_bill_no not in withdraw_refs:
missing_withdraw += 1
has_refund = refund_counts.get(order.out_bill_no, 0) > 0
if order.status in {"failed", "rejected"} and not has_refund:
missing_refund += 1
if has_refund and order.status not in {"failed", "rejected"}:
refund_on_non_terminal += 1
return {
"missing_withdraw": missing_withdraw,
"missing_refund": missing_refund,
"duplicate_refund": sum(1 for count in refund_counts.values() if count > 1),
"refund_on_non_terminal": refund_on_non_terminal,
}
def withdraw_ledger_check(db: Session) -> dict:
"""现金账本校验:两本物理隔离的账各自对账(产品红线:coin_cash / invite_cash 不串)。
普通现金:CoinAccount.cash_balance_cents ↔ cash_transaction(withdraw/withdraw_refund);
邀请奖励金:CoinAccount.invite_cash_balance_cents ↔ invite_cash_transaction
(invite_withdraw/invite_withdraw_refund)。
提现单按 source 分流到对应账核对——邀请提现的流水写在 invite_cash_transaction 表,
绝不能拿去和普通现金流水比(否则每笔邀请提现单都会被误报「缺扣款/缺退款流水」)。
分流口径与 create_withdraw 一致:仅 source==invite_cash 走邀请账,其余(含历史空值)归普通现金。
"""
orders = list(db.execute(select(WithdrawOrder)).scalars().all())
coin_orders = [o for o in orders if o.source != "invite_cash"]
invite_orders = [o for o in orders if o.source == "invite_cash"]
# —— 普通现金账(coin_cash) ——
cash_balance_total = int(
db.execute(select(func.coalesce(func.sum(CoinAccount.cash_balance_cents), 0))).scalar_one()
)
cash_txn_total = int(
db.execute(select(func.coalesce(func.sum(CashTransaction.amount_cents), 0))).scalar_one()
)
cash_txns = list(
db.execute(
select(CashTransaction).where(
CashTransaction.biz_type.in_(("withdraw", "withdraw_refund"))
)
).scalars().all()
)
coin = _check_withdraw_ledger_side(
coin_orders, cash_txns, withdraw_biz="withdraw", refund_biz="withdraw_refund"
)
cash_diff = cash_balance_total - cash_txn_total
# —— 邀请奖励金账(invite_cash,独立账户 + 独立流水表) ——
invite_balance_total = int(
db.execute(
select(func.coalesce(func.sum(CoinAccount.invite_cash_balance_cents), 0))
).scalar_one()
)
invite_txn_total = int(
db.execute(
select(func.coalesce(func.sum(InviteCashTransaction.amount_cents), 0))
).scalar_one()
)
invite_txns = list(
db.execute(
select(InviteCashTransaction).where(
InviteCashTransaction.biz_type.in_(("invite_withdraw", "invite_withdraw_refund"))
)
).scalars().all()
)
invite = _check_withdraw_ledger_side(
invite_orders, invite_txns,
withdraw_biz="invite_withdraw", refund_biz="invite_withdraw_refund",
)
invite_diff = invite_balance_total - invite_txn_total
ok = (
cash_diff == 0
and invite_diff == 0
and all(v == 0 for v in coin.values())
and all(v == 0 for v in invite.values())
)
return {
"ok": ok,
# 普通现金账(coin_cash:金币兑换的现金)
"cash_balance_total_cents": cash_balance_total,
"cash_transaction_total_cents": cash_txn_total,
"balance_diff_cents": cash_diff,
"missing_withdraw_txn_count": coin["missing_withdraw"],
"missing_refund_txn_count": coin["missing_refund"],
"duplicate_refund_txn_count": coin["duplicate_refund"],
"refund_txn_on_non_terminal_count": coin["refund_on_non_terminal"],
# 邀请奖励金账(invite_cash:与普通现金物理隔离,各自对账)
"invite_cash_balance_total_cents": invite_balance_total,
"invite_cash_transaction_total_cents": invite_txn_total,
"invite_balance_diff_cents": invite_diff,
"invite_missing_withdraw_txn_count": invite["missing_withdraw"],
"invite_missing_refund_txn_count": invite["missing_refund"],
"invite_duplicate_refund_txn_count": invite["duplicate_refund"],
"invite_refund_txn_on_non_terminal_count": invite["refund_on_non_terminal"],
}
def get_user_overview(db: Session, user_id: int) -> dict | None:
"""用户 360 概览:基础资料 + 钱包余额 + 各项 count。历史明细走各自分页接口(带 user_id 过滤)。"""
user = db.get(User, user_id)
@@ -1147,50 +1257,112 @@ def user_reward_stats(
*,
date_from: datetime | None = None,
date_to: datetime | None = None,
withdraw_source: str | None = None,
app_env: str | None = None,
revenue_scope: str = "all",
feed_scene: str | None = None,
) -> dict:
"""提现详情「用户统计区」10 项。窗口作用于除「现金余额」外的所有项(余额是当前快照)。
口径:激励视频/信息流只统计 granted;数量——视频按条数、信息流按份数(unit_count 累加);
平均 eCPM 用原始分值(分/千次)按记录取算术平均;各「提现」= 该来源累计金币折现。
口径:激励视频/信息流奖励数量只统计 granted;数量——视频按条数、信息流按份数(unit_count 累加)
平均 Draw eCPM 与广告收益报表一致:基于 ad_ecpm_record 的全部 draw/feed 展示记录计算,
不以是否发奖为筛选条件。各「提现」= 该来源累计金币折现。
传统任务 = 窗口内正向金币中,排除广告(reward_video/feed_ad_reward)与人工调整后的折现。
"""
withdraw_source_conds = (
[WithdrawOrder.source == withdraw_source] if withdraw_source else []
)
wd_success = db.execute(
select(func.coalesce(func.sum(WithdrawOrder.amount_cents), 0)).where(
WithdrawOrder.user_id == user_id,
WithdrawOrder.status == "success",
*withdraw_source_conds,
*_window_conds(WithdrawOrder.created_at, date_from, date_to),
)
).scalar_one()
wd_total = db.execute(
select(func.count(WithdrawOrder.id)).where(
WithdrawOrder.user_id == user_id,
*withdraw_source_conds,
*_window_conds(WithdrawOrder.created_at, date_from, date_to),
)
).scalar_one()
acc = db.get(CoinAccount, user_id) # 现金余额:当前快照,不随窗口
cash_balance = acc.cash_balance_cents if acc else 0
rv = list(db.execute(
select(AdRewardRecord).where(
AdRewardRecord.user_id == user_id,
AdRewardRecord.reward_scene == "reward_video",
AdRewardRecord.status == "granted",
*_window_conds(AdRewardRecord.created_at, date_from, date_to),
cash_balance = (
(
acc.invite_cash_balance_cents
if withdraw_source == "invite_cash"
else acc.cash_balance_cents
)
).scalars())
if acc
else 0
)
# 只投影本统计实际使用的列。避免滚动发布或旧本地库尚未补齐无关新列时,
# SQLAlchemy 因 select(ORM) 自动展开整表字段而让提现详情整体 500。
business_ids: set[str] | None = None
if revenue_scope == "business":
# 与广告收益报表共用正式/测试业务代码位集合,避免两个页面随配置切换后再次漂移。
from app.admin.repositories.ad_revenue import business_code_ids
business_ids = business_code_ids(db, app_env)
rv_conds = [
AdRewardRecord.user_id == user_id,
AdRewardRecord.reward_scene == "reward_video",
AdRewardRecord.status == "granted",
*_window_conds(AdRewardRecord.created_at, date_from, date_to),
]
if app_env is not None:
rv_conds.append(AdRewardRecord.app_env == app_env)
if business_ids is not None:
rv_conds.append(AdRewardRecord.our_code_id.in_(business_ids))
rv = db.execute(
select(AdRewardRecord.ecpm_raw, AdRewardRecord.coin).where(
*rv_conds,
)
).all()
rv_ecpms = [rewards.parse_ecpm_fen(r.ecpm_raw) for r in rv if r.ecpm_raw]
rv_coins = sum(r.coin for r in rv)
feed = list(db.execute(
select(AdFeedRewardRecord).where(
AdFeedRewardRecord.user_id == user_id,
AdFeedRewardRecord.status == "granted",
*_window_conds(AdFeedRewardRecord.created_at, date_from, date_to),
feed_reward_conds = [
AdFeedRewardRecord.user_id == user_id,
AdFeedRewardRecord.status == "granted",
*_window_conds(AdFeedRewardRecord.created_at, date_from, date_to),
]
if app_env is not None:
feed_reward_conds.append(AdFeedRewardRecord.app_env == app_env)
if feed_scene is not None:
feed_reward_conds.append(AdFeedRewardRecord.feed_scene == feed_scene)
if business_ids is not None:
feed_reward_conds.append(AdFeedRewardRecord.our_code_id.in_(business_ids))
feed_rewards = db.execute(
select(
AdFeedRewardRecord.unit_count,
AdFeedRewardRecord.coin,
).where(
*feed_reward_conds,
)
).scalars())
feed_ecpms = [rewards.parse_ecpm_fen(f.ecpm_raw) for f in feed if f.ecpm_raw]
feed_coins = sum(f.coin for f in feed)
).all()
feed_coins = sum(f.coin for f in feed_rewards)
feed_impression_conds = [
AdEcpmRecord.user_id == user_id,
AdEcpmRecord.ad_type.in_(("draw", "feed")),
*_window_conds(AdEcpmRecord.created_at, date_from, date_to),
]
if app_env is not None:
feed_impression_conds.append(AdEcpmRecord.app_env == app_env)
if feed_scene is not None:
feed_impression_conds.append(AdEcpmRecord.feed_scene == feed_scene)
if business_ids is not None:
feed_impression_conds.append(AdEcpmRecord.our_code_id.in_(business_ids))
feed_impressions = db.execute(
select(AdEcpmRecord.ecpm_raw).where(*feed_impression_conds)
).all()
# 与 ad_revenue.category_stats 相同:每次展示权重相同,非法原值按 parse_ecpm_fen 记 0。
feed_ecpms = [rewards.parse_ecpm_fen(row.ecpm_raw) for row in feed_impressions]
trad_coins = db.execute(
select(func.coalesce(func.sum(CoinTransaction.amount), 0)).where(
@@ -1209,7 +1381,7 @@ def user_reward_stats(
"reward_video_count": len(rv),
"reward_video_avg_ecpm": round(sum(rv_ecpms) / len(rv_ecpms), 2) if rv_ecpms else 0.0,
"reward_video_cash_cents": _coins_to_cents(rv_coins),
"feed_count": int(sum(f.unit_count for f in feed)),
"feed_count": int(sum(f.unit_count for f in feed_rewards)),
"feed_avg_ecpm": round(sum(feed_ecpms) / len(feed_ecpms), 2) if feed_ecpms else 0.0,
"feed_cash_cents": _coins_to_cents(feed_coins),
}
@@ -1222,6 +1394,11 @@ def _cn_wall_to_utc(dt: datetime) -> datetime:
return dt.replace(tzinfo=rewards.CN_TZ).astimezone(timezone.utc).replace(tzinfo=None)
def _coin_record_sort_key(row: dict) -> datetime:
"""金币明细跨数据源排序键:兼容 SQLite naive 与 PostgreSQL aware 时间。"""
return _as_utc(row["created_at"])
def user_coin_records(
db: Session,
user_id: int,
@@ -1246,8 +1423,14 @@ def user_coin_records(
signin_from = date_from + timedelta(hours=8) if date_from is not None else None
signin_to = date_to + timedelta(hours=8) if date_to is not None else None
# 三类来源都只取页面需要的列,避免无关 ORM 新列造成旧库查询失败。
for rec in db.execute(
select(AdRewardRecord)
select(
AdRewardRecord.reward_scene,
AdRewardRecord.created_at,
AdRewardRecord.ecpm_raw,
AdRewardRecord.coin,
)
.where(
AdRewardRecord.user_id == user_id,
AdRewardRecord.status == "granted",
@@ -1255,7 +1438,7 @@ def user_coin_records(
)
.order_by(AdRewardRecord.created_at.desc())
.limit(fetch)
).scalars():
).all():
is_video = rec.reward_scene == "reward_video"
rows.append({
"source": rec.reward_scene,
@@ -1266,7 +1449,12 @@ def user_coin_records(
})
for rec in db.execute(
select(AdFeedRewardRecord)
select(
AdFeedRewardRecord.feed_scene,
AdFeedRewardRecord.created_at,
AdFeedRewardRecord.ecpm_raw,
AdFeedRewardRecord.coin,
)
.where(
AdFeedRewardRecord.user_id == user_id,
AdFeedRewardRecord.status == "granted",
@@ -1274,7 +1462,7 @@ def user_coin_records(
)
.order_by(AdFeedRewardRecord.created_at.desc())
.limit(fetch)
).scalars():
).all():
rows.append({
"source": "feed",
"source_label": _FEED_SCENE_LABEL.get(rec.feed_scene, "信息流广告"),
@@ -1284,7 +1472,7 @@ def user_coin_records(
})
for rec in db.execute(
select(CoinTransaction)
select(CoinTransaction.created_at, CoinTransaction.amount)
.where(
CoinTransaction.user_id == user_id,
CoinTransaction.biz_type == "signin",
@@ -1292,7 +1480,7 @@ def user_coin_records(
)
.order_by(CoinTransaction.created_at.desc())
.limit(fetch)
).scalars():
).all():
rows.append({
"source": "signin",
"source_label": "签到",
@@ -1302,7 +1490,10 @@ def user_coin_records(
"coin": rec.amount,
})
rows.sort(key=lambda r: r["created_at"], reverse=True)
# SQLite 常返回 naive datetimePostgreSQL timestamptz 返回 aware datetime
# 统一成 aware UTC 排序,避免线上合并广告记录与签到记录时抛
# “can't compare offset-naive and offset-aware datetimes”。
rows.sort(key=_coin_record_sort_key, reverse=True)
has_more = len(rows) > offset + limit
# 总数 = 三源在窗口内 granted 计数之和(供前端页码分页渲染页码/共 N 条)
+545
View File
@@ -0,0 +1,545 @@
"""风控监控读模型:聚合通用行为流水、风险事件与比价记录。"""
from __future__ import annotations
from datetime import UTC, datetime, timedelta
from sqlalchemy import desc, func, select
from sqlalchemy.orm import Session
from app.core.config import settings
from app.models.comparison import ComparisonRecord
from app.models.device import DeviceLiveness
from app.models.risk import BehaviorEvent, RiskIncident
from app.models.user import User
from app.repositories import risk as risk_repo
KIND_TO_RULE = {
"sms": risk_repo.RULE_SMS_HOURLY,
"oneclick": risk_repo.RULE_ONECLICK_DAILY,
"compare": risk_repo.RULE_COMPARE_DAILY,
}
RULE_TO_KIND = {rule: kind for kind, rule in KIND_TO_RULE.items()}
def _day_bounds(now: datetime | None = None) -> tuple[str, datetime, datetime]:
current = (now or datetime.now(UTC)).astimezone(risk_repo.CN_TZ)
start = current.replace(hour=0, minute=0, second=0, microsecond=0)
end = start + timedelta(days=1)
return (
start.strftime("%Y-%m-%d"),
start.astimezone(UTC),
end.astimezone(UTC),
)
def _compare_day_bounds(now: datetime | None = None) -> tuple[datetime, datetime]:
current = (now or datetime.now(UTC)).astimezone(risk_repo.CN_TZ).replace(
tzinfo=None
)
start = current.replace(hour=0, minute=0, second=0, microsecond=0)
return start, start + timedelta(days=1)
def sync_today_compare_incidents(db: Session, now: datetime | None = None) -> None:
risk_repo.reconcile_compare_rule(db, at=now)
def reset_open_alerts(
db: Session, *, admin_id: int, reset_at: datetime
) -> dict[str, int]:
"""将三类待处理报警归零,并从 reset_at 开始重新累计。"""
rule_codes = tuple(KIND_TO_RULE.values())
incidents = list(
db.scalars(
select(RiskIncident).where(
RiskIncident.rule_code.in_(rule_codes),
RiskIncident.status == "open",
)
).all()
)
counts = {kind: 0 for kind in KIND_TO_RULE}
for incident in incidents:
incident.status = "resolved"
incident.action_reason = risk_repo.MANUAL_RESET_REASON
incident.handled_by = admin_id
incident.handled_at = reset_at
counts[RULE_TO_KIND[incident.rule_code]] += 1
risk_repo.reset_rule_baselines(
db,
rule_codes=rule_codes,
reset_at=reset_at,
admin_id=admin_id,
)
return counts
def summary(db: Session, now: datetime | None = None) -> dict:
date_key, start, end = _day_bounds(now)
compare_start, compare_end = _compare_day_bounds(now)
current = now or datetime.now(UTC)
risk_repo.reconcile_behavior_rule(
db, rule_code=risk_repo.RULE_SMS_HOURLY, at=current, commit=False
)
risk_repo.reconcile_behavior_rule(
db, rule_code=risk_repo.RULE_ONECLICK_DAILY, at=current, commit=False
)
risk_repo.reconcile_compare_rule(db, at=current, commit=False)
db.commit()
sms_threshold = risk_repo.get_rule_threshold(db, risk_repo.RULE_SMS_HOURLY)
oneclick_threshold = risk_repo.get_rule_threshold(
db, risk_repo.RULE_ONECLICK_DAILY
)
compare_threshold = risk_repo.get_rule_threshold(db, risk_repo.RULE_COMPARE_DAILY)
def _behavior_total(event_type: str, outcomes: tuple[str, ...]) -> int:
return int(
db.scalar(
select(func.count(BehaviorEvent.id)).where(
BehaviorEvent.event_type == event_type,
BehaviorEvent.outcome.in_(outcomes),
BehaviorEvent.occurred_at >= start,
BehaviorEvent.occurred_at < end,
)
)
or 0
)
compare_total = int(
db.scalar(
select(func.count(ComparisonRecord.id)).where(
ComparisonRecord.created_at >= compare_start,
ComparisonRecord.created_at < compare_end,
)
)
or 0
)
def _open_count(rule_code: str) -> int:
return int(
db.scalar(
select(func.count(RiskIncident.id)).where(
RiskIncident.rule_code == rule_code,
RiskIncident.status == "open",
RiskIncident.window_key.like(f"{date_key}%"),
)
)
or 0
)
return {
"date": date_key,
"updated_at": now or datetime.now(UTC),
"cards": [
{
"kind": "sms",
"alert_subject_count": _open_count(risk_repo.RULE_SMS_HOURLY),
"today_total": _behavior_total(risk_repo.EVENT_SMS_SEND, ("success",)),
"threshold": sms_threshold,
"rule_text": f"单设备 1 小时 ≥ {sms_threshold}",
},
{
"kind": "oneclick",
"alert_subject_count": _open_count(risk_repo.RULE_ONECLICK_DAILY),
"today_total": _behavior_total(
risk_repo.EVENT_ONECLICK_LOGIN, ("success", "failed")
),
"threshold": oneclick_threshold,
"rule_text": f"单设备当日 ≥ {oneclick_threshold}",
},
{
"kind": "compare",
"alert_subject_count": _open_count(risk_repo.RULE_COMPARE_DAILY),
"today_total": compare_total,
"threshold": compare_threshold,
"rule_text": f"单账户当日 ≥ {compare_threshold}",
},
],
}
def _latest_device_model(db: Session, subject_id: str) -> str | None:
return db.execute(
select(BehaviorEvent.device_model)
.where(
BehaviorEvent.subject_type == "device",
BehaviorEvent.subject_id == subject_id,
BehaviorEvent.device_model.is_not(None),
BehaviorEvent.device_model != "",
)
.order_by(BehaviorEvent.occurred_at.desc(), BehaviorEvent.id.desc())
.limit(1)
).scalar_one_or_none()
def _first_used_at(db: Session, subject_id: str) -> datetime | None:
behavior_first = db.scalar(
select(func.min(BehaviorEvent.occurred_at)).where(
BehaviorEvent.subject_type == "device",
BehaviorEvent.subject_id == subject_id,
)
)
registered_first = db.scalar(
select(func.min(DeviceLiveness.created_at)).where(
DeviceLiveness.device_id == subject_id
)
)
observed = [value for value in (behavior_first, registered_first) if value is not None]
if not observed:
return None
def _utc(value: datetime) -> datetime:
return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC)
return min(observed, key=_utc)
def _common_device(db: Session, user_id: int, start: datetime, end: datetime) -> str | None:
return db.execute(
select(ComparisonRecord.device_id)
.where(
ComparisonRecord.user_id == user_id,
ComparisonRecord.device_id.is_not(None),
ComparisonRecord.created_at >= start,
ComparisonRecord.created_at < end,
)
.group_by(ComparisonRecord.device_id)
.order_by(func.count(ComparisonRecord.id).desc(), ComparisonRecord.device_id.asc())
.limit(1)
).scalar_one_or_none()
def list_incidents(
db: Session,
*,
kind: str,
status: str = "open",
limit: int = 20,
cursor: int = 0,
) -> tuple[list[dict], int | None, int]:
rule_code = KIND_TO_RULE[kind]
stmt = select(RiskIncident).where(RiskIncident.rule_code == rule_code)
count_stmt = select(func.count(RiskIncident.id)).where(
RiskIncident.rule_code == rule_code
)
if status != "all":
stmt = stmt.where(RiskIncident.status == status)
count_stmt = count_stmt.where(RiskIncident.status == status)
total = int(db.scalar(count_stmt) or 0)
incidents = list(
db.scalars(
stmt.order_by(desc(RiskIncident.triggered_at), desc(RiskIncident.id))
.offset(cursor)
.limit(limit)
).all()
)
user_ids = [int(i.subject_id) for i in incidents if i.subject_type == "user"]
users = {
u.id: u
for u in db.scalars(select(User).where(User.id.in_(user_ids))).all()
} if user_ids else {}
items: list[dict] = []
for incident in incidents:
item = {
"incident_id": incident.id,
"kind": kind,
"subject_type": incident.subject_type,
"subject_id": incident.subject_id,
"window_start": incident.window_start,
"window_end": incident.window_end,
"first_event_at": incident.first_event_at,
"triggered_at": incident.triggered_at,
"last_event_at": incident.last_event_at,
"event_count": incident.event_count,
"status": incident.status,
}
if incident.subject_type == "device":
restriction = risk_repo.get_active_restriction(
db,
subject_type="device",
subject_id=incident.subject_id,
scope=risk_repo.SCOPE_AUTH_DEVICE,
)
item["device_model"] = _latest_device_model(db, incident.subject_id)
item["first_used_at"] = _first_used_at(db, incident.subject_id)
item["restricted"] = restriction is not None
item["restriction_id"] = restriction.id if restriction else None
else:
uid = int(incident.subject_id)
user = users.get(uid)
restriction = risk_repo.get_active_restriction(
db,
subject_type="user",
subject_id=incident.subject_id,
scope=risk_repo.SCOPE_ECONOMIC_ACCOUNT,
)
item.update(
{
"user_id": uid,
"phone": user.phone if user else None,
"registered_at": user.created_at if user else None,
"common_device_id": _common_device(
db, uid, incident.window_start, incident.window_end
),
"restricted": restriction is not None,
"restriction_id": restriction.id if restriction else None,
}
)
items.append(item)
next_cursor = cursor + len(items) if cursor + len(items) < total else None
return items, next_cursor, total
def _page_events(
db: Session,
incident: RiskIncident,
*,
event_type: str,
outcomes: tuple[str, ...] | None,
limit: int,
cursor: int,
) -> tuple[list[BehaviorEvent], int]:
filters = [
BehaviorEvent.event_type == event_type,
BehaviorEvent.subject_type == incident.subject_type,
BehaviorEvent.subject_id == incident.subject_id,
BehaviorEvent.occurred_at >= incident.window_start,
BehaviorEvent.occurred_at < incident.window_end,
]
if outcomes:
filters.append(BehaviorEvent.outcome.in_(outcomes))
total = int(db.scalar(select(func.count(BehaviorEvent.id)).where(*filters)) or 0)
rows = list(
db.scalars(
select(BehaviorEvent)
.where(*filters)
.order_by(BehaviorEvent.occurred_at.desc(), BehaviorEvent.id.desc())
.offset(cursor)
.limit(limit)
).all()
)
return rows, total
def _user_map(db: Session, user_ids: set[int]) -> dict[int, User]:
if not user_ids:
return {}
return {u.id: u for u in db.scalars(select(User).where(User.id.in_(user_ids))).all()}
def _distinct_event_users(
db: Session,
incident: RiskIncident,
*,
event_type: str,
outcomes: tuple[str, ...],
) -> int:
return int(
db.scalar(
select(func.count(func.distinct(BehaviorEvent.user_id))).where(
BehaviorEvent.event_type == event_type,
BehaviorEvent.subject_type == incident.subject_type,
BehaviorEvent.subject_id == incident.subject_id,
BehaviorEvent.outcome.in_(outcomes),
BehaviorEvent.user_id.is_not(None),
BehaviorEvent.occurred_at >= incident.window_start,
BehaviorEvent.occurred_at < incident.window_end,
)
)
or 0
)
def _sms_distinct_accounts(db: Session, incident: RiskIncident) -> int:
"""统计整个告警窗口内,确实完成本设备短信验证的不同账号数。"""
send_rows = db.execute(
select(BehaviorEvent.phone, BehaviorEvent.occurred_at).where(
BehaviorEvent.event_type == risk_repo.EVENT_SMS_SEND,
BehaviorEvent.subject_type == incident.subject_type,
BehaviorEvent.subject_id == incident.subject_id,
BehaviorEvent.outcome == "success",
BehaviorEvent.phone.is_not(None),
BehaviorEvent.occurred_at >= incident.window_start,
BehaviorEvent.occurred_at < incident.window_end,
)
).all()
phones = {phone for phone, _ in send_rows}
if not phones:
return 0
login_rows = db.execute(
select(
BehaviorEvent.user_id,
BehaviorEvent.phone,
BehaviorEvent.occurred_at,
)
.where(
BehaviorEvent.event_type == risk_repo.EVENT_SMS_LOGIN,
BehaviorEvent.subject_id == incident.subject_id,
BehaviorEvent.phone.in_(phones),
BehaviorEvent.user_id.is_not(None),
BehaviorEvent.outcome == "success",
BehaviorEvent.occurred_at >= incident.window_start,
BehaviorEvent.occurred_at
< incident.window_end + timedelta(seconds=settings.SMS_CODE_TTL_SEC),
)
.order_by(BehaviorEvent.occurred_at.asc(), BehaviorEvent.id.asc())
).all()
matched_users = {
user_id
for user_id, phone, login_at in login_rows
if any(
sent_phone == phone
and sent_at <= login_at
and login_at
<= sent_at + timedelta(seconds=settings.SMS_CODE_TTL_SEC)
for sent_phone, sent_at in send_rows
)
}
return len(matched_users)
def incident_details(
db: Session,
*,
incident: RiskIncident,
kind: str,
limit: int,
cursor: int,
) -> dict:
if kind == "sms":
rows, total = _page_events(
db,
incident,
event_type=risk_repo.EVENT_SMS_SEND,
outcomes=("success",),
limit=limit,
cursor=cursor,
)
phone_values = {r.phone for r in rows if r.phone}
login_rows = list(
db.scalars(
select(BehaviorEvent)
.where(
BehaviorEvent.event_type == risk_repo.EVENT_SMS_LOGIN,
BehaviorEvent.subject_id == incident.subject_id,
BehaviorEvent.phone.in_(phone_values),
BehaviorEvent.outcome == "success",
BehaviorEvent.occurred_at >= incident.window_start,
BehaviorEvent.occurred_at
< incident.window_end + timedelta(seconds=settings.SMS_CODE_TTL_SEC),
)
.order_by(BehaviorEvent.occurred_at.asc(), BehaviorEvent.id.asc())
).all()
) if phone_values else []
users = _user_map(db, {r.user_id for r in login_rows if r.user_id is not None})
items = []
for row in rows:
match = next(
(
login
for login in login_rows
if login.phone == row.phone
and login.occurred_at >= row.occurred_at
and login.occurred_at
<= row.occurred_at + timedelta(seconds=settings.SMS_CODE_TTL_SEC)
),
None,
)
user = users.get(match.user_id) if match and match.user_id else None
items.append(
{
"id": row.id,
"occurred_at": row.occurred_at,
"outcome": row.outcome,
"phone": row.phone,
"user_id": user.id if user else None,
"username": user.username if user else None,
"account_registered_at": user.created_at if user else None,
"verified": match is not None,
}
)
distinct_accounts = _sms_distinct_accounts(db, incident)
elif kind == "oneclick":
rows, total = _page_events(
db,
incident,
event_type=risk_repo.EVENT_ONECLICK_LOGIN,
outcomes=("success", "failed"),
limit=limit,
cursor=cursor,
)
users = _user_map(db, {r.user_id for r in rows if r.user_id is not None})
items = []
for row in rows:
user = users.get(row.user_id) if row.user_id else None
items.append(
{
"id": row.id,
"occurred_at": row.occurred_at,
"outcome": row.outcome,
"phone": row.phone,
"user_id": row.user_id,
"username": user.username if user else None,
"account_registered_at": user.created_at if user else None,
"reason": row.reason,
}
)
distinct_accounts = _distinct_event_users(
db,
incident,
event_type=risk_repo.EVENT_ONECLICK_LOGIN,
outcomes=("success", "failed"),
)
else:
user_id = int(incident.subject_id)
filters = (
ComparisonRecord.user_id == user_id,
ComparisonRecord.created_at >= incident.window_start,
ComparisonRecord.created_at < incident.window_end,
)
total = int(db.scalar(select(func.count(ComparisonRecord.id)).where(*filters)) or 0)
rows = list(
db.scalars(
select(ComparisonRecord)
.where(*filters)
.order_by(ComparisonRecord.created_at.desc(), ComparisonRecord.id.desc())
.offset(cursor)
.limit(limit)
).all()
)
items = []
for row in rows:
prices = {
str(result.get("platform_name") or result.get("platform_id") or "未知平台"): (
float(result["price"]) if result.get("price") is not None else None
)
for result in (row.comparison_results or [])
}
items.append(
{
"id": row.id,
"occurred_at": row.created_at,
"outcome": row.status,
"store_name": row.store_name,
"product_names": row.product_names,
"prices": prices,
"saved_amount_yuan": (
row.saved_amount_cents / 100
if row.saved_amount_cents is not None
else None
),
"status": row.status,
}
)
distinct_accounts = None
next_cursor = cursor + len(items) if cursor + len(items) < total else None
return {
"kind": kind,
"incident_id": incident.id,
"event_count": incident.event_count,
"distinct_accounts": distinct_accounts,
"items": items,
"next_cursor": next_cursor,
"total": total,
}
+3 -4
View File
@@ -30,18 +30,17 @@ from app.models.user import User
from app.models.wallet import CoinTransaction, WithdrawOrder
_BEIJING = timezone(timedelta(hours=8))
REWARD_VIDEO_BIZ_TYPES = ("reward_video", "ad_reward")
REWARD_VIDEO_BIZ_TYPES = ("reward_video", "ad_reward", "signin_boost")
# 领券/比价奖励金币的真实来源是信息流广告发奖(ad_feed_reward_record,按 feed_scene 分场景);
# coin_transaction 里只有扁平的 feed_ad_reward、biz_type 不分 coupon/comparison,故这俩桶历史从未
# 被写入,仅留作未来兜底,实际金额在下方按 feed_scene 汇总 ad_feed_reward_record 得出。reward_video/
# ad_reward 是激励视频,单独成桶、不再混进领券奖励(历史误并会把激励视频金币双计进领券)
# 被写入,仅留作未来兜底,实际金额在下方按 feed_scene 汇总 ad_feed_reward_record 得出。
# reward_video/ad_reward 及历史 signin_boost 均归看视频桶,不再混进领券奖励或常规任务
COUPON_REWARD_BIZ_TYPES = ("coupon", "coupon_reward")
COMPARISON_REWARD_BIZ_TYPES = ("comparison", "compare_reward", "comparison_reward")
# 常规任务必须按明确来源相加;不能从全部正向流水反减排除项,否则新增广告/运营
# biz_type 时会在排除清单更新前自动混入该桶。task_ 前缀在查询处单独覆盖现有及未来任务。
REGULAR_TASK_EXACT_BIZ_TYPES = (
"signin",
"signin_boost",
"price_report_reward",
"feedback_reward",
)
+6
View File
@@ -30,6 +30,12 @@ def _validate(key: str, value: Any) -> None:
if t == "int":
if not isinstance(value, int) or isinstance(value, bool) or value < 0:
raise ValueError("需为非负整数")
minimum = CONFIG_DEFS[key].get("min")
maximum = CONFIG_DEFS[key].get("max")
if minimum is not None and value < minimum:
raise ValueError(f"不能小于 {minimum}")
if maximum is not None and value > maximum:
raise ValueError(f"不能大于 {maximum}")
elif t == "int_list":
if not isinstance(value, list) or not value:
raise ValueError("需为非空整数列表")
+167 -85
View File
@@ -12,6 +12,10 @@ from app.admin.repositories import mutations, queries
from app.admin.schemas.common import CursorPage, OkResponse
from app.admin.schemas.feedback import (
FeedbackApproveRequest,
FeedbackBulkApproveRequest,
FeedbackBulkItemResult,
FeedbackBulkRejectRequest,
FeedbackBulkResult,
FeedbackOut,
FeedbackRejectRequest,
FeedbackSummary,
@@ -33,6 +37,123 @@ def _ensure_pending(fb: Feedback) -> None:
raise HTTPException(status_code=400, detail="反馈已审核")
def _approve_feedback(
db: AdminDb,
admin: AdminUser,
feedback_id: int,
payload: FeedbackApproveRequest | FeedbackBulkApproveRequest,
ip: str,
*,
bulk: bool = False,
) -> FeedbackOut:
fb = db.get(Feedback, feedback_id, with_for_update=True)
if fb is None:
raise HTTPException(status_code=404, detail="反馈不存在")
_ensure_pending(fb)
before = fb.status
mutations.review_feedback(
db,
fb,
status="adopted",
reward_coins=payload.reward_coins,
review_note=payload.note,
admin_reply=payload.reply,
reviewed_by_admin_id=admin.id,
commit=False,
)
wallet_repo.grant_coins(
db,
fb.user_id,
payload.reward_coins,
biz_type="feedback_reward",
ref_id=str(fb.id),
remark="意见反馈被采纳",
)
detail = {
"before": before,
"after": "adopted",
"reward_coins": payload.reward_coins,
"note": payload.note,
"reply": payload.reply,
}
if bulk:
detail["bulk"] = True
write_audit(
db,
admin,
action="feedback.approve",
target_type="feedback",
target_id=feedback_id,
detail=detail,
ip=ip,
commit=False,
)
db.commit()
db.refresh(fb)
out = FeedbackOut.model_validate(fb)
notification_events.notify_feedback_reward(db, fb)
return out
def _reject_feedback(
db: AdminDb,
admin: AdminUser,
feedback_id: int,
payload: FeedbackRejectRequest | FeedbackBulkRejectRequest,
ip: str,
*,
bulk: bool = False,
) -> FeedbackOut:
fb = db.get(Feedback, feedback_id, with_for_update=True)
if fb is None:
raise HTTPException(status_code=404, detail="反馈不存在")
_ensure_pending(fb)
before = fb.status
mutations.review_feedback(
db,
fb,
status="rejected",
reject_reason=payload.reason,
review_note=payload.note,
admin_reply=payload.reply,
reviewed_by_admin_id=admin.id,
commit=False,
)
detail = {
"before": before,
"after": "rejected",
"reason": payload.reason,
"note": payload.note,
"reply": payload.reply,
}
if bulk:
detail["bulk"] = True
write_audit(
db,
admin,
action="feedback.reject",
target_type="feedback",
target_id=feedback_id,
detail=detail,
ip=ip,
commit=False,
)
db.commit()
db.refresh(fb)
out = FeedbackOut.model_validate(fb)
notification_events.notify_feedback_reply(db, fb)
return out
def _bulk_result(items: list[FeedbackBulkItemResult]) -> FeedbackBulkResult:
success = sum(1 for item in items if item.ok)
return FeedbackBulkResult(
total=len(items), success=success, failed=len(items) - success, items=items,
)
@router.get("", response_model=CursorPage[FeedbackOut], summary="反馈工单列表")
def list_feedbacks(
db: AdminDb,
@@ -73,6 +194,50 @@ def feedback_summary(db: AdminDb) -> FeedbackSummary:
return FeedbackSummary.model_validate(queries.feedback_summary(db))
@router.post("/bulk/approve", response_model=FeedbackBulkResult, summary="批量采纳反馈并发金币")
def bulk_approve_feedbacks(
body: FeedbackBulkApproveRequest,
request: Request,
admin: Annotated[AdminUser, Depends(require_role("operator"))],
db: AdminDb,
) -> FeedbackBulkResult:
results: list[FeedbackBulkItemResult] = []
ip = get_client_ip(request)
for feedback_id in body.ids:
try:
out = _approve_feedback(db, admin, feedback_id, body, ip, bulk=True)
results.append(FeedbackBulkItemResult(id=feedback_id, ok=True, status=out.status))
except HTTPException as exc:
db.rollback()
results.append(FeedbackBulkItemResult(id=feedback_id, ok=False, error=str(exc.detail)))
except Exception: # noqa: BLE001 - 单笔失败不打断整批
db.rollback()
results.append(FeedbackBulkItemResult(id=feedback_id, ok=False, error="系统异常"))
return _bulk_result(results)
@router.post("/bulk/reject", response_model=FeedbackBulkResult, summary="批量拒绝采纳反馈")
def bulk_reject_feedbacks(
body: FeedbackBulkRejectRequest,
request: Request,
admin: Annotated[AdminUser, Depends(require_role("operator"))],
db: AdminDb,
) -> FeedbackBulkResult:
results: list[FeedbackBulkItemResult] = []
ip = get_client_ip(request)
for feedback_id in body.ids:
try:
out = _reject_feedback(db, admin, feedback_id, body, ip, bulk=True)
results.append(FeedbackBulkItemResult(id=feedback_id, ok=True, status=out.status))
except HTTPException as exc:
db.rollback()
results.append(FeedbackBulkItemResult(id=feedback_id, ok=False, error=str(exc.detail)))
except Exception: # noqa: BLE001 - 单笔失败不打断整批
db.rollback()
results.append(FeedbackBulkItemResult(id=feedback_id, ok=False, error="系统异常"))
return _bulk_result(results)
@router.post("/{feedback_id}/handle", response_model=OkResponse, summary="标记反馈已处理")
def handle_feedback(
feedback_id: int,
@@ -93,53 +258,7 @@ def approve_feedback(
admin: Annotated[AdminUser, Depends(require_role("operator"))],
db: AdminDb,
) -> FeedbackOut:
fb = db.get(Feedback, feedback_id)
if fb is None:
raise HTTPException(status_code=404, detail="反馈不存在")
_ensure_pending(fb)
before = fb.status
mutations.review_feedback(
db,
fb,
status="adopted",
reward_coins=payload.reward_coins,
review_note=payload.note,
admin_reply=payload.reply,
reviewed_by_admin_id=admin.id,
commit=False,
)
wallet_repo.grant_coins(
db,
fb.user_id,
payload.reward_coins,
biz_type="feedback_reward",
ref_id=str(fb.id),
remark="意见反馈被采纳",
)
write_audit(
db,
admin,
action="feedback.approve",
target_type="feedback",
target_id=feedback_id,
detail={
"before": before,
"after": "adopted",
"reward_coins": payload.reward_coins,
"note": payload.note,
"reply": payload.reply,
},
ip=get_client_ip(request),
commit=False,
)
db.commit()
db.refresh(fb)
out = FeedbackOut.model_validate(fb)
# PRD #10 反馈奖励:采纳发金币后通知用户(站内 + push,必带官方留言)。
# 业务已 commit,通知失败只 log 不影响审核结果。
notification_events.notify_feedback_reward(db, fb)
return out
return _approve_feedback(db, admin, feedback_id, payload, get_client_ip(request))
@router.post("/{feedback_id}/reject", response_model=FeedbackOut, summary="拒绝采纳反馈")
@@ -150,41 +269,4 @@ def reject_feedback(
admin: Annotated[AdminUser, Depends(require_role("operator"))],
db: AdminDb,
) -> FeedbackOut:
fb = db.get(Feedback, feedback_id)
if fb is None:
raise HTTPException(status_code=404, detail="反馈不存在")
_ensure_pending(fb)
before = fb.status
mutations.review_feedback(
db,
fb,
status="rejected",
reject_reason=payload.reason,
review_note=payload.note,
admin_reply=payload.reply,
reviewed_by_admin_id=admin.id,
commit=False,
)
write_audit(
db,
admin,
action="feedback.reject",
target_type="feedback",
target_id=feedback_id,
detail={
"before": before,
"after": "rejected",
"reason": payload.reason,
"note": payload.note,
"reply": payload.reply,
},
ip=get_client_ip(request),
commit=False,
)
db.commit()
db.refresh(fb)
out = FeedbackOut.model_validate(fb)
# PRD #9 官方回复:未采纳也回复了用户(原因/留言用户端可见),通知去反馈历史页查看。
notification_events.notify_feedback_reply(db, fb)
return out
return _reject_feedback(db, admin, feedback_id, payload, get_client_ip(request))
+101
View File
@@ -0,0 +1,101 @@
"""admin 新手引导视频配置:读 / 改开关次数金币 / 上传视频 / 删视频(带审计)。
整份配置存通用 app_config 表(见 app/repositories/guide_video.py),App 领券等候浮层
每次展示前调 POST /api/v1/guide-video/start 同步。权限:operator 可改(运营维护),
super 恒可;读为只读(任意已登录 admin)。
⚠️ 视频上限 100MB(settings.GUIDE_VIDEO_MAX_BYTES),已在 admin nginx 为本接口单独放宽
client_max_body_size,见 shaguabijia-admin-web/deploy/nginx/admin.shaguabijia.com.conf。
"""
from __future__ import annotations
from typing import Annotated
from fastapi import APIRouter, Depends, File, HTTPException, Request, UploadFile
from app.admin.audit import write_audit
from app.admin.deps import AdminDb, get_client_ip, get_current_admin, require_role
from app.admin.schemas.guide_video import GuideVideoConfigOut, GuideVideoConfigUpdate
from app.core import media
from app.models.admin import AdminUser
from app.repositories import guide_video
router = APIRouter(
prefix="/admin/api/guide-video",
tags=["admin-guide-video"],
dependencies=[Depends(get_current_admin)],
)
def _out(db: AdminDb) -> GuideVideoConfigOut:
"""配置 + 播放统计合成响应(四个写接口都以最新状态返回,前端一次同步到位)。"""
return GuideVideoConfigOut(**guide_video.get_config(db), **guide_video.play_stats(db))
@router.get("", response_model=GuideVideoConfigOut, summary="新手引导视频配置(领券浮层)")
def get_config(db: AdminDb) -> GuideVideoConfigOut:
return _out(db)
@router.patch("", response_model=GuideVideoConfigOut, summary="改开关/次数/金币(带审计)")
def update_config(
body: GuideVideoConfigUpdate,
request: Request,
admin: Annotated[AdminUser, Depends(require_role("operator"))],
db: AdminDb,
) -> GuideVideoConfigOut:
before, after = guide_video.update_config(
db,
enabled=body.enabled,
max_plays=body.max_plays,
reward_coin=body.reward_coin,
admin_id=admin.id,
commit=False,
)
write_audit(
db, admin, action="guide_video.update", target_type="guide_video", target_id=None,
detail={"before": before, "after": after}, ip=get_client_ip(request), commit=False,
)
db.commit()
return _out(db)
@router.post("/video", response_model=GuideVideoConfigOut, summary="上传新手引导视频(MP4,带审计)")
async def upload_video(
request: Request,
admin: Annotated[AdminUser, Depends(require_role("operator"))],
db: AdminDb,
file: UploadFile = File(...),
) -> GuideVideoConfigOut:
data = await file.read()
try:
url = media.save_guide_video(data)
except media.MediaError as e:
raise HTTPException(status_code=400, detail=str(e)) from e
before, after = guide_video.set_video(db, url, admin_id=admin.id, commit=False)
write_audit(
db, admin, action="guide_video.set_video", target_type="guide_video", target_id=None,
detail={"before": before.get("video_url"), "after": url, "bytes": len(data)},
ip=get_client_ip(request), commit=False,
)
db.commit()
# 提交成功后再删旧片,避免新片没落库就把旧片丢了
media.delete_guide_video(before.get("video_url"))
return _out(db)
@router.delete("/video", response_model=GuideVideoConfigOut, summary="移除新手引导视频(带审计)")
def delete_video(
request: Request,
admin: Annotated[AdminUser, Depends(require_role("operator"))],
db: AdminDb,
) -> GuideVideoConfigOut:
"""移除后 /guide-video/start 一律返回 should_play=false,领券浮层回到「只放广告」。"""
before, after = guide_video.set_video(db, None, admin_id=admin.id, commit=False)
write_audit(
db, admin, action="guide_video.delete_video", target_type="guide_video", target_id=None,
detail={"before": before.get("video_url")}, ip=get_client_ip(request), commit=False,
)
db.commit()
media.delete_guide_video(before.get("video_url"))
return _out(db)
+103 -33
View File
@@ -17,6 +17,10 @@ from app.admin.deps import AdminDb, get_client_ip, get_current_admin, require_ro
from app.admin.repositories import mutations, queries
from app.admin.schemas.common import CursorPage, OkResponse
from app.admin.schemas.price_report import (
PriceReportBulkItemResult,
PriceReportBulkRejectRequest,
PriceReportBulkRequest,
PriceReportBulkResult,
PriceReportOut,
PriceReportRejectRequest,
PriceReportSummary,
@@ -34,6 +38,59 @@ router = APIRouter(
)
def _approve_price_report(
db: AdminDb, admin: AdminUser, report_id: int, ip: str, *, bulk: bool = False
) -> PriceReport:
rep = db.get(PriceReport, report_id, with_for_update=True)
if rep is None:
raise HTTPException(status_code=404, detail="上报记录不存在")
if rep.status != "pending":
raise HTTPException(status_code=400, detail=f"该上报已审核过(当前 {rep.status}),不可重复操作")
coins = PRICE_REPORT_REWARD_COINS
mutations.review_price_report(db, rep, status="approved", reward_coins=coins, commit=False)
wallet_repo.grant_coins(
db, rep.user_id, coins,
biz_type="price_report_reward", ref_id=str(rep.id), remark="上报更低价审核通过",
)
detail = {"reward_coins": coins, "user_id": rep.user_id}
if bulk:
detail["bulk"] = True
write_audit(
db, admin, action="price_report.approve", target_type="price_report", target_id=report_id,
detail=detail, ip=ip, commit=False,
)
db.commit()
notification_events.notify_report_approved(db, rep)
return rep
def _reject_price_report(
db: AdminDb, admin: AdminUser, report_id: int, reason: str, ip: str, *, bulk: bool = False
) -> PriceReport:
rep = db.get(PriceReport, report_id, with_for_update=True)
if rep is None:
raise HTTPException(status_code=404, detail="上报记录不存在")
if rep.status != "pending":
raise HTTPException(status_code=400, detail=f"该上报已审核过(当前 {rep.status}),不可重复操作")
mutations.review_price_report(db, rep, status="rejected", reject_reason=reason, commit=False)
detail = {"reason": reason, "user_id": rep.user_id}
if bulk:
detail["bulk"] = True
write_audit(
db, admin, action="price_report.reject", target_type="price_report", target_id=report_id,
detail=detail, ip=ip, commit=False,
)
db.commit()
return rep
def _bulk_result(items: list[PriceReportBulkItemResult]) -> PriceReportBulkResult:
success = sum(1 for item in items if item.ok)
return PriceReportBulkResult(
total=len(items), success=success, failed=len(items) - success, items=items,
)
@router.get("", response_model=CursorPage[PriceReportOut], summary="上报更低价列表(筛选+分页)")
def list_price_reports(
db: AdminDb,
@@ -60,6 +117,50 @@ def price_report_summary(db: AdminDb) -> PriceReportSummary:
return PriceReportSummary.model_validate(queries.price_report_summary(db))
@router.post("/bulk/approve", response_model=PriceReportBulkResult, summary="批量通过上报(发固定金币)")
def bulk_approve_price_reports(
body: PriceReportBulkRequest,
request: Request,
admin: Annotated[AdminUser, Depends(require_role("operator"))],
db: AdminDb,
) -> PriceReportBulkResult:
results: list[PriceReportBulkItemResult] = []
ip = get_client_ip(request)
for report_id in body.ids:
try:
rep = _approve_price_report(db, admin, report_id, ip, bulk=True)
results.append(PriceReportBulkItemResult(id=report_id, ok=True, status=rep.status))
except HTTPException as exc:
db.rollback()
results.append(PriceReportBulkItemResult(id=report_id, ok=False, error=str(exc.detail)))
except Exception: # noqa: BLE001 - 单笔失败不打断整批
db.rollback()
results.append(PriceReportBulkItemResult(id=report_id, ok=False, error="系统异常"))
return _bulk_result(results)
@router.post("/bulk/reject", response_model=PriceReportBulkResult, summary="批量拒绝上报")
def bulk_reject_price_reports(
body: PriceReportBulkRejectRequest,
request: Request,
admin: Annotated[AdminUser, Depends(require_role("operator"))],
db: AdminDb,
) -> PriceReportBulkResult:
results: list[PriceReportBulkItemResult] = []
ip = get_client_ip(request)
for report_id in body.ids:
try:
rep = _reject_price_report(db, admin, report_id, body.reason, ip, bulk=True)
results.append(PriceReportBulkItemResult(id=report_id, ok=True, status=rep.status))
except HTTPException as exc:
db.rollback()
results.append(PriceReportBulkItemResult(id=report_id, ok=False, error=str(exc.detail)))
except Exception: # noqa: BLE001 - 单笔失败不打断整批
db.rollback()
results.append(PriceReportBulkItemResult(id=report_id, ok=False, error="系统异常"))
return _bulk_result(results)
@router.post("/{report_id}/approve", response_model=OkResponse, summary="通过上报(发固定金币)")
def approve_price_report(
report_id: int,
@@ -67,27 +168,7 @@ def approve_price_report(
admin: Annotated[AdminUser, Depends(require_role("operator"))],
db: AdminDb,
) -> OkResponse:
# 行锁(SELECT FOR UPDATE):并发/连点双请求会都读到 pending → 各发一次金币双倍发奖,
# 锁住该行串行化,第二个请求拿锁后看到 approved → 走 400。SQLite 下 FOR UPDATE 为 no-op。
rep = db.get(PriceReport, report_id, with_for_update=True)
if rep is None:
raise HTTPException(status_code=404, detail="上报记录不存在")
if rep.status != "pending":
raise HTTPException(status_code=400, detail=f"该上报已审核过(当前 {rep.status}),不可重复操作")
coins = PRICE_REPORT_REWARD_COINS
# 改状态 + 发金币 + 审计同一事务(commit=False),最后一起 commit:改了就有痕、发了就留账
mutations.review_price_report(db, rep, status="approved", reward_coins=coins, commit=False)
wallet_repo.grant_coins(
db, rep.user_id, coins,
biz_type="price_report_reward", ref_id=str(rep.id), remark="上报更低价审核通过",
)
write_audit(
db, admin, action="price_report.approve", target_type="price_report", target_id=report_id,
detail={"reward_coins": coins, "user_id": rep.user_id}, ip=get_client_ip(request), commit=False,
)
db.commit()
# PRD #11 爆料审核通过:发金币后通知用户(站内 + push)。业务已 commit,通知失败只 log。
notification_events.notify_report_approved(db, rep)
_approve_price_report(db, admin, report_id, get_client_ip(request))
return OkResponse()
@@ -99,16 +180,5 @@ def reject_price_report(
admin: Annotated[AdminUser, Depends(require_role("operator"))],
db: AdminDb,
) -> OkResponse:
rep = db.get(PriceReport, report_id, with_for_update=True) # 行锁,同 approve(防并发重复审核)
if rep is None:
raise HTTPException(status_code=404, detail="上报记录不存在")
if rep.status != "pending":
raise HTTPException(status_code=400, detail=f"该上报已审核过(当前 {rep.status}),不可重复操作")
reason = body.reason.strip()
mutations.review_price_report(db, rep, status="rejected", reject_reason=reason, commit=False)
write_audit(
db, admin, action="price_report.reject", target_type="price_report", target_id=report_id,
detail={"reason": reason, "user_id": rep.user_id}, ip=get_client_ip(request), commit=False,
)
db.commit()
_reject_price_report(db, admin, report_id, body.reason, get_client_ip(request))
return OkResponse()
+331
View File
@@ -0,0 +1,331 @@
"""风控监控:统计、风险事件列表/详情及忽略、封禁、解封。"""
from __future__ import annotations
from typing import Annotated, Literal
from fastapi import APIRouter, Depends, HTTPException, Query, Request, status
from app.admin.audit import write_audit
from app.admin.deps import (
AdminDb,
get_client_ip,
require_page,
require_role,
)
from app.admin.repositories import risk_monitor as repo
from app.admin.schemas.risk_monitor import (
RestrictionRevokeResponse,
RiskActionRequest,
RiskActionResponse,
RiskDetailPage,
RiskIncidentPage,
RiskMonitorSummary,
RiskResetResponse,
RiskRuleConfig,
)
from app.core.config_schema import (
RISK_COMPARE_DAILY_THRESHOLD_KEY,
RISK_ONECLICK_DAILY_THRESHOLD_KEY,
RISK_SMS_HOURLY_THRESHOLD_KEY,
)
from app.models.risk import RiskIncident, SubjectRestriction
from app.repositories import app_config
from app.repositories import risk as risk_repo
router = APIRouter(
prefix="/admin/api/risk-monitor",
tags=["admin-risk-monitor"],
dependencies=[Depends(require_page("risk-monitor"))],
)
RiskKind = Literal["sms", "oneclick", "compare"]
def _rule_config(db) -> RiskRuleConfig:
return RiskRuleConfig(
sms_hourly_threshold=risk_repo.get_rule_threshold(
db, risk_repo.RULE_SMS_HOURLY
),
oneclick_daily_threshold=risk_repo.get_rule_threshold(
db, risk_repo.RULE_ONECLICK_DAILY
),
compare_daily_threshold=risk_repo.get_rule_threshold(
db, risk_repo.RULE_COMPARE_DAILY
),
)
def _incident_or_404(db, incident_id: int, kind: RiskKind | None = None) -> RiskIncident:
incident = db.get(RiskIncident, incident_id)
if incident is None:
raise HTTPException(status_code=404, detail="风险事件不存在")
if kind and incident.rule_code != repo.KIND_TO_RULE[kind]:
raise HTTPException(status_code=404, detail="风险事件类型不匹配")
return incident
@router.get("/summary", response_model=RiskMonitorSummary, summary="风控监控顶部卡片")
def get_summary(db: AdminDb) -> RiskMonitorSummary:
return RiskMonitorSummary(**repo.summary(db))
@router.get("/rules", response_model=RiskRuleConfig, summary="读取风控规则阈值")
def get_rules(db: AdminDb) -> RiskRuleConfig:
return _rule_config(db)
@router.patch("/rules", response_model=RiskRuleConfig, summary="修改风控规则并即时重算")
def update_rules(
body: RiskRuleConfig,
request: Request,
admin: Annotated[object, Depends(require_role("operator", "tech"))],
db: AdminDb,
) -> RiskRuleConfig:
before = _rule_config(db).model_dump()
after = body.model_dump()
values = (
(RISK_SMS_HOURLY_THRESHOLD_KEY, body.sms_hourly_threshold),
(RISK_ONECLICK_DAILY_THRESHOLD_KEY, body.oneclick_daily_threshold),
(RISK_COMPARE_DAILY_THRESHOLD_KEY, body.compare_daily_threshold),
)
for key, value in values:
app_config.set_value(
db, key, value, admin_id=admin.id, commit=False
)
now = risk_repo.utcnow()
risk_repo.reconcile_behavior_rule(
db, rule_code=risk_repo.RULE_SMS_HOURLY, at=now, commit=False
)
risk_repo.reconcile_behavior_rule(
db, rule_code=risk_repo.RULE_ONECLICK_DAILY, at=now, commit=False
)
risk_repo.reconcile_compare_rule(db, at=now, commit=False)
write_audit(
db,
admin,
action="risk.rules.update",
target_type="risk_rule",
target_id="all",
detail={"before": before, "after": after},
ip=get_client_ip(request),
commit=False,
)
db.commit()
return _rule_config(db)
@router.post(
"/reset",
response_model=RiskResetResponse,
summary="重置三类待处理报警并从零重新累计",
)
def reset_alerts(
request: Request,
admin: Annotated[object, Depends(require_role("operator", "tech"))],
db: AdminDb,
) -> RiskResetResponse:
reset_at = risk_repo.utcnow()
counts = repo.reset_open_alerts(db, admin_id=admin.id, reset_at=reset_at)
total = sum(counts.values())
write_audit(
db,
admin,
action="risk.alerts.reset",
target_type="risk_monitor",
target_id="all",
detail={
"reset_at": reset_at.isoformat(),
"reset_incident_count": total,
"reset_counts": counts,
"restrictions_changed": False,
},
ip=get_client_ip(request),
commit=False,
)
db.commit()
return RiskResetResponse(
reset_at=reset_at,
reset_incident_count=total,
reset_counts=counts,
)
@router.get(
"/incidents/{kind}",
response_model=RiskIncidentPage,
summary="按短信/一键登录/比价读取风险事件",
)
def get_incidents(
kind: RiskKind,
db: AdminDb,
incident_status: Annotated[
Literal["open", "ignored", "blocked", "resolved", "all"], Query(alias="status")
] = "open",
limit: Annotated[int, Query(ge=1, le=100)] = 20,
cursor: Annotated[int, Query(ge=0)] = 0,
) -> RiskIncidentPage:
if kind == "compare":
repo.sync_today_compare_incidents(db)
items, next_cursor, total = repo.list_incidents(
db, kind=kind, status=incident_status, limit=limit, cursor=cursor
)
return RiskIncidentPage(items=items, next_cursor=next_cursor, total=total)
@router.get(
"/incidents/{kind}/{incident_id}/details",
response_model=RiskDetailPage,
summary="风险事件当期事实明细",
)
def get_incident_details(
kind: RiskKind,
incident_id: int,
db: AdminDb,
limit: Annotated[int, Query(ge=1, le=100)] = 20,
cursor: Annotated[int, Query(ge=0)] = 0,
) -> RiskDetailPage:
incident = _incident_or_404(db, incident_id, kind)
return RiskDetailPage(
**repo.incident_details(
db, incident=incident, kind=kind, limit=limit, cursor=cursor
)
)
@router.post(
"/incidents/{incident_id}/ignore",
response_model=RiskActionResponse,
summary="忽略风险事件",
)
def ignore_incident(
incident_id: int,
body: RiskActionRequest,
request: Request,
admin: Annotated[object, Depends(require_role("operator", "tech"))],
db: AdminDb,
) -> RiskActionResponse:
incident = _incident_or_404(db, incident_id)
if incident.status != "open":
raise HTTPException(status_code=409, detail="该风险事件已处理")
incident.status = "ignored"
incident.action_reason = body.reason.strip() or "管理员确认忽略"
incident.handled_by = admin.id
incident.handled_at = risk_repo.utcnow()
write_audit(
db,
admin,
action="risk.incident.ignore",
target_type="risk_incident",
target_id=incident.id,
detail={
"rule_code": incident.rule_code,
"subject_type": incident.subject_type,
"subject_id": incident.subject_id,
"reason": incident.action_reason,
},
ip=get_client_ip(request),
commit=False,
)
db.commit()
return RiskActionResponse(incident_id=incident.id, status=incident.status)
@router.post(
"/incidents/{incident_id}/block",
response_model=RiskActionResponse,
summary="封禁风险主体",
)
def block_incident(
incident_id: int,
body: RiskActionRequest,
request: Request,
admin: Annotated[object, Depends(require_role("operator", "tech"))],
db: AdminDb,
) -> RiskActionResponse:
incident = _incident_or_404(db, incident_id)
if incident.status not in ("open", "blocked"):
raise HTTPException(status_code=409, detail="该风险事件已处理")
if incident.subject_type == "device":
if incident.subject_id.startswith("legacy-ip:"):
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="旧客户端未上报设备ID,不能执行设备封禁",
)
scope = risk_repo.SCOPE_AUTH_DEVICE
elif incident.subject_type == "user":
scope = risk_repo.SCOPE_ECONOMIC_ACCOUNT
else: # pragma: no cover - 当前三条规则只会生成 device/user
raise HTTPException(status_code=400, detail="该主体类型暂不支持封禁")
reason = body.reason.strip() or "风控规则命中"
restriction = risk_repo.activate_restriction(
db,
subject_type=incident.subject_type,
subject_id=incident.subject_id,
scope=scope,
reason=reason,
incident_id=incident.id,
admin_id=admin.id,
)
incident.status = "blocked"
incident.action_reason = reason
incident.handled_by = admin.id
incident.handled_at = risk_repo.utcnow()
write_audit(
db,
admin,
action="risk.subject.block",
target_type=incident.subject_type,
target_id=incident.subject_id,
detail={
"incident_id": incident.id,
"rule_code": incident.rule_code,
"scope": scope,
"reason": reason,
},
ip=get_client_ip(request),
commit=False,
)
db.commit()
return RiskActionResponse(
incident_id=incident.id,
status=incident.status,
restriction_id=restriction.id,
)
@router.post(
"/restrictions/{restriction_id}/revoke",
response_model=RestrictionRevokeResponse,
summary="解除主体限制",
)
def revoke_restriction(
restriction_id: int,
request: Request,
admin: Annotated[object, Depends(require_role("operator", "tech"))],
db: AdminDb,
) -> RestrictionRevokeResponse:
restriction = db.get(SubjectRestriction, restriction_id)
if restriction is None:
raise HTTPException(status_code=404, detail="限制记录不存在")
if restriction.active:
risk_repo.revoke_restriction(db, restriction=restriction, admin_id=admin.id)
if restriction.incident_id:
incident = db.get(RiskIncident, restriction.incident_id)
if incident and incident.status == "blocked":
incident.status = "resolved"
incident.handled_by = admin.id
incident.handled_at = risk_repo.utcnow()
write_audit(
db,
admin,
action="risk.subject.unblock",
target_type=restriction.subject_type,
target_id=restriction.subject_id,
detail={"restriction_id": restriction.id, "scope": restriction.scope},
ip=get_client_ip(request),
commit=False,
)
db.commit()
return RestrictionRevokeResponse(restriction_id=restriction.id)
+64 -2
View File
@@ -16,6 +16,7 @@ from app.admin.schemas.user import (
GrantCashRequest,
GrantCoinsRequest,
SetDebugTraceRequest,
SetUserRiskRequest,
SetUserStatusRequest,
UserCoinRecord,
UserRewardStats,
@@ -84,12 +85,29 @@ def get_user_reward_stats(
db: AdminDb,
date_from: Annotated[datetime | None, Query()] = None,
date_to: Annotated[datetime | None, Query()] = None,
withdraw_source: Annotated[
str | None, Query(pattern="^(coin_cash|invite_cash)$")
] = None,
app_env: Annotated[str | None, Query(pattern="^(prod|test)$")] = None,
revenue_scope: Annotated[str, Query(pattern="^(business|all)$")] = "all",
feed_scene: Annotated[
str | None, Query(pattern="^(comparison|coupon|welfare)$")
] = None,
) -> UserRewardStats:
"""提现详情抽屉「用户统计区」。date_from/date_to 都不传 = 注册至今(全量)。"""
if user_repo.get_user_by_id(db, user_id) is None:
if not user_repo.user_exists(db, user_id):
raise HTTPException(status_code=404, detail="用户不存在")
return UserRewardStats(
**queries.user_reward_stats(db, user_id, date_from=date_from, date_to=date_to)
**queries.user_reward_stats(
db,
user_id,
date_from=date_from,
date_to=date_to,
withdraw_source=withdraw_source,
app_env=app_env,
revenue_scope=revenue_scope,
feed_scene=feed_scene,
)
)
@@ -138,6 +156,50 @@ def set_user_status(
return OkResponse()
@router.post("/{user_id}/risk", response_model=OkResponse, summary="设置人工高风险标记与备注")
def set_user_risk(
user_id: int,
body: SetUserRiskRequest,
request: Request,
admin: Annotated[
AdminUser, Depends(require_role("operator", "finance"))
],
db: AdminDb,
) -> OkResponse:
user = user_repo.get_user_by_id(db, user_id)
if user is None:
raise HTTPException(status_code=404, detail="用户不存在")
before = {
"is_high_risk": user.is_high_risk,
"high_risk_note": user.high_risk_note,
}
mutations.set_user_risk(
db,
user,
is_high_risk=body.is_high_risk,
note=body.note,
commit=False,
)
write_audit(
db,
admin,
action="user.risk.set",
target_type="user",
target_id=user_id,
detail={
"before": before,
"after": {
"is_high_risk": user.is_high_risk,
"high_risk_note": user.high_risk_note,
},
},
ip=get_client_ip(request),
commit=False,
)
db.commit()
return OkResponse()
@router.post("/{user_id}/debug-trace", response_model=OkResponse, summary="开关调试链接权限")
def set_user_debug_trace(
user_id: int,
+49 -38
View File
@@ -1,4 +1,4 @@
"""admin 提现:列表(读)+ 单笔重试查单 + 批量对账(写,带审计)。
"""admin 提现:列表(读)+ 单笔/批量查单与审核操作(写,带审计)。
提现的钱逻辑(查微信/退款/撤单/幂等)全部复用 app.repositories.wallet,admin 只触发 + 记审计。
这些 wallet 函数内部各自 commit(涉及微信调用),审计在其后单独 commit:操作本身幂等,
@@ -14,17 +14,16 @@ from fastapi import APIRouter, Depends, HTTPException, Query, Request
from app.admin.audit import write_audit
from app.admin.deps import AdminDb, get_client_ip, get_current_admin, require_role
from app.admin.repositories import queries
from app.admin.schemas.common import CursorPage
from app.admin.schemas.admin import AdminAuditLogOut
from app.admin.schemas.common import CursorPage
from app.admin.schemas.wallet import (
CashTxnOut,
ReconcileResult,
InviteOverviewOut,
WithdrawBulkItemResult,
WithdrawBulkRejectRequest,
WithdrawBulkRequest,
WithdrawBulkResult,
WithdrawBulkItemResult,
WithdrawDetailOut,
WithdrawLedgerCheckOut,
WithdrawListItemOut,
WithdrawOrderOut,
WithdrawRejectRequest,
@@ -83,8 +82,16 @@ def list_withdraws(
cursor=cursor,
)
# 联表带出手机号/昵称 + 累计成功提现(本页 user_id 批量富化,2 条聚合查询,无 N+1)。
enrichment = queries.withdraw_list_enrichment(db, [o.user_id for o in items])
_empty = {"phone": None, "nickname": None, "cumulative_success_cents": 0}
enrichment = queries.withdraw_list_enrichment(
db, [o.user_id for o in items], source=source
)
_empty = {
"phone": None,
"nickname": None,
"is_high_risk": False,
"high_risk_note": None,
"cumulative_success_cents": 0,
}
out_items = [
WithdrawListItemOut.model_validate(o).model_copy(
update=enrichment.get(o.user_id, _empty)
@@ -95,8 +102,13 @@ def list_withdraws(
@router.get("/summary", response_model=WithdrawSummaryOut, summary="提现审核台统计")
def withdraws_summary(db: AdminDb) -> WithdrawSummaryOut:
return WithdrawSummaryOut(**queries.withdraw_summary(db))
def withdraws_summary(
db: AdminDb,
source: Annotated[
str | None, Query(pattern="^(coin_cash|invite_cash)$")
] = None,
) -> WithdrawSummaryOut:
return WithdrawSummaryOut(**queries.withdraw_summary(db, source=source))
@router.get(
@@ -129,13 +141,11 @@ def withdraw_health_check(db: AdminDb) -> WxpayHealthCheckOut:
issues.append("免确认授权回调地址未配置")
# 实际是否自动对账 = env 部署总闸(worker 起没起)AND 运营后台 DB 开关(本轮跑不跑)。
# 自动查单属于非阻断运维能力:状态继续返回给调用方,但关闭时不计入微信提现配置 issues,
# 避免把“没有自动扫单”误报成“无法打款”。
worker_running = settings.WITHDRAW_AUTO_RECONCILE_ENABLED
daily_on = bool(app_config.get_value(db, "withdraw_auto_reconcile_enabled"))
auto_reconcile_enabled = worker_running and daily_on
if not worker_running:
issues.append("自动对账 worker 未启动(部署侧 env WITHDRAW_AUTO_RECONCILE_ENABLED=false)")
elif not daily_on:
issues.append("自动对账运营开关已关闭(系统配置页可开)")
return WxpayHealthCheckOut(
ok=not issues,
@@ -155,13 +165,13 @@ def withdraw_health_check(db: AdminDb) -> WxpayHealthCheckOut:
)
@router.get("/ledger-check", response_model=WithdrawLedgerCheckOut, summary="提现资金账本校验")
def withdraw_ledger_check(db: AdminDb) -> WithdrawLedgerCheckOut:
return WithdrawLedgerCheckOut(**queries.withdraw_ledger_check(db))
@router.get("/{out_bill_no}", response_model=WithdrawDetailOut, summary="提现单详情")
def withdraw_detail(out_bill_no: str, db: AdminDb) -> WithdrawDetailOut:
def withdraw_detail(
out_bill_no: str,
db: AdminDb,
date_from: Annotated[datetime | None, Query()] = None,
date_to: Annotated[datetime | None, Query()] = None,
) -> WithdrawDetailOut:
order = queries.get_withdraw_by_out_bill_no(db, out_bill_no)
if order is None:
raise HTTPException(status_code=404, detail="提现单不存在")
@@ -174,6 +184,8 @@ def withdraw_detail(out_bill_no: str, db: AdminDb) -> WithdrawDetailOut:
id=user.id,
phone=user.phone,
nickname=user.nickname,
is_high_risk=user.is_high_risk,
high_risk_note=user.high_risk_note,
status=user.status,
wechat_nickname=user.wechat_nickname,
wechat_avatar_url=user.wechat_avatar_url,
@@ -197,37 +209,36 @@ def withdraw_detail(out_bill_no: str, db: AdminDb) -> WithdrawDetailOut:
recent_withdraws,
overview["cash_balance_cents"] if overview else 0,
)
detail_enrichment = queries.withdraw_list_enrichment(
db, [order.user_id], source=order.source
).get(order.user_id, {})
return WithdrawDetailOut(
order=WithdrawOrderOut.model_validate(order),
user=user_snapshot,
cumulative_success_cents=int(
detail_enrichment.get("cumulative_success_cents", 0)
),
risk_flags=risk_flags,
risk_score=risk_score,
recent_withdraws=[WithdrawOrderOut.model_validate(o) for o in recent_withdraws],
recent_cash_transactions=[CashTxnOut.model_validate(t) for t in recent_cash_transactions],
audit_logs=[AdminAuditLogOut.model_validate(log) for log in audit_logs],
invite_overview=(
InviteOverviewOut(
**queries.invite_overview(
db,
order.user_id,
date_from=date_from,
date_to=date_to,
)
)
if order.source == "invite_cash"
else None
),
)
# 注意:/reconcile 必须在 /{out_bill_no}/refresh 之前声明(静态路径优先于路径参数)
@router.post("/reconcile", response_model=ReconcileResult, summary="批量对账(扫超时 pending 单)")
def reconcile(
request: Request,
admin: Annotated[AdminUser, Depends(require_role("finance"))],
db: AdminDb,
older_than_minutes: Annotated[int, Query(ge=0)] = 15,
) -> ReconcileResult:
try:
result = wallet_repo.reconcile_pending_withdraws(db, older_than_minutes=older_than_minutes)
except wxpay.WxPayNotConfiguredError as e:
raise HTTPException(status_code=503, detail="微信支付未配置") from e
write_audit(
db, admin, action="withdraw.reconcile", target_type="withdraw", target_id=None,
detail=result, ip=get_client_ip(request), commit=True,
)
return ReconcileResult(**result)
def _bulk_result(items: list[WithdrawBulkItemResult]) -> WithdrawBulkResult:
success = sum(1 for item in items if item.ok)
return WithdrawBulkResult(
+7
View File
@@ -40,6 +40,8 @@ class AdRevenueRecord(BaseModel):
expected_coin: int = Field(..., description="按公式复算应发金币")
actual_coin: int = Field(..., description="实际入账金币")
matched: bool = Field(..., description="复算与实发是否一致")
adn: str | None = Field(None, description="本条发奖对应的实际填充 ADN 子渠道")
slot_id: str | None = Field(None, description="本条发奖对应的底层 mediation rit")
class AdRevenueDaily(BaseModel):
@@ -73,6 +75,7 @@ class AdRevenueTypeStat(BaseModel):
impressions: int = Field(..., description="该类型展示条数合计")
revenue_yuan: float = Field(..., description="该类型预估收益合计(元)")
ecpm_yuan: float | None = Field(None, description="按真实展示次数加权的 SDK eCPM(元/千次)")
class AdRevenueRow(BaseModel):
@@ -142,6 +145,10 @@ class AdRevenueReportOut(BaseModel):
default_factory=dict,
description="按广告类型(ad_type)小计 {ad_type: {impressions, revenue_yuan}};前端取 draw / reward_video 做分类大盘",
)
category_stats: dict[str, AdRevenueTypeStat] = Field(
default_factory=dict,
description="按经营分类小计:draw=draw+历史 feedvideo=reward_video+withdrawal_video",
)
scene_stats: dict[str, AdRevenueTypeStat] = Field(
default_factory=dict,
description="按信息流场景(feed_scene)小计 {comparison/coupon/welfare: {impressions, revenue_yuan}};"
+3 -1
View File
@@ -29,6 +29,7 @@ class AdminComparisonListItem(BaseModel):
source_price_cents: int | None = None
best_price_cents: int | None = None
saved_amount_cents: int | None = None
ordered: bool = False
# debug 概览
total_ms: int | None = None
step_count: int | None = None
@@ -39,8 +40,10 @@ class AdminComparisonListItem(BaseModel):
# 本次比价 LLM 总成本(元,按当时价冻结);旧记录/未回填为 None → 前端「成本」列回退估算。见 services/llm_cost.py。
llm_cost_yuan: float | None = None
device_model: str | None = None
device_model_name: str | None = None
rom_vendor: str | None = None
rom_name: str | None = None
rom_version: int | None = None
android_version: str | None = None
app_version: str | None = None
ad_revenue_yuan: float = 0.0 # 本次比价看的信息流广告预估收益(元),queries 瞬态挂 ORM 实例上
@@ -84,7 +87,6 @@ class AdminComparisonDetail(AdminComparisonListItem):
skipped_dish_names: list = []
# 全量环境
device_manufacturer: str | None = None
rom_version: int | None = None
android_sdk: int | None = None
app_version_code: int | None = None
source_app_version: str | None = None
+2 -2
View File
@@ -79,10 +79,10 @@ class CouponDataRow(BaseModel):
started_at: datetime = Field(..., description="发起时刻(明细「时间」列)")
claimed_count: int | None = None
point_success_count: int | None = Field(
None, description="本次成功券点位数(success+already_claimed);无逐券埋点为空"
None, description="本次成功券数(success+already_claimed);无逐券事件为空"
)
point_total_count: int | None = Field(
None, description="本次尝试券点位数(success+already_claimed+failed,不含 skipped);无逐券埋点为空"
None, description="本次尝试券数(success+already_claimed+failed,不含 skipped);无逐券事件为空"
)
trace_url: str | None = Field(None, description="pricebot 公网 trace 链接(仅 completed 有);admin 渲染可点链接,无则显示可复制 trace_id")
ad_revenue_yuan: float = Field(
+52 -1
View File
@@ -3,7 +3,7 @@ from __future__ import annotations
from datetime import datetime
from pydantic import BaseModel, ConfigDict, Field
from pydantic import BaseModel, ConfigDict, Field, field_validator
from app.core.rewards import FEEDBACK_REWARD_MAX_COINS
@@ -32,7 +32,10 @@ class FeedbackOut(BaseModel):
# 提交端环境快照(feedback 表列):提交版本号 / 机型OS版本;改版前的历史反馈为 None
app_version: str | None = None
device_model: str | None = None
device_model_name: str | None = None
device_manufacturer: str | None = None
rom_name: str | None = None
rom_version: int | None = None
android_version: str | None = None
# 联表瞬态字段(queries._attach_user_info 挂):列表展示完整手机号,点手机号查该用户全部反馈
phone: str | None = None
@@ -55,6 +58,54 @@ class FeedbackRejectRequest(BaseModel):
reply: str | None = Field(default=None, max_length=256, description="给用户的回复留言,用户端可见")
class FeedbackBulkRequest(BaseModel):
ids: list[int] = Field(min_length=1, max_length=50, description="待审核反馈 ID 列表")
@field_validator("ids")
@classmethod
def _ids_must_be_unique(cls, ids: list[int]) -> list[int]:
if len(ids) != len(set(ids)):
raise ValueError("反馈 ID 不能重复")
return ids
class FeedbackBulkApproveRequest(FeedbackBulkRequest):
reward_coins: int = Field(
ge=1,
le=FEEDBACK_REWARD_MAX_COINS,
description="每条采纳反馈发放的金币数",
)
note: str | None = Field(default=None, max_length=256, description="采纳要点/审核备注(内部)")
reply: str | None = Field(default=None, max_length=256, description="给用户的回复留言,用户端可见")
class FeedbackBulkRejectRequest(FeedbackBulkRequest):
reason: str = Field(min_length=1, max_length=256, description="批量未采纳原因,用户端可见")
note: str | None = Field(default=None, max_length=256, description="运营内部审核备注")
reply: str | None = Field(default=None, max_length=256, description="给用户的回复留言,用户端可见")
@field_validator("reason")
@classmethod
def _reason_not_blank(cls, value: str) -> str:
if not value.strip():
raise ValueError("未采纳原因不能为空")
return value.strip()
class FeedbackBulkItemResult(BaseModel):
id: int
ok: bool
status: str | None = None
error: str | None = None
class FeedbackBulkResult(BaseModel):
total: int
success: int
failed: int
items: list[FeedbackBulkItemResult]
class FeedbackSummary(BaseModel):
"""审核台顶部各状态计数(pending 含历史 new 态)。"""
+25
View File
@@ -0,0 +1,25 @@
"""admin 新手引导视频配置 schemas(开关 / 视频地址 / 前几次 / 每次金币)。"""
from __future__ import annotations
from pydantic import BaseModel, Field
from app.repositories.guide_video import MAX_PLAYS_LIMIT, REWARD_COIN_LIMIT
class GuideVideoConfigOut(BaseModel):
enabled: bool
video_url: str | None = None # 相对地址 /media/guide_video/xxx.mp4;未配片 = None
max_plays: int
reward_coin: int
updated_at: str | None = None
# 只读统计,后台展示用:已有多少次播放、其中已发币多少次。
total_plays: int = 0
granted_plays: int = 0
class GuideVideoConfigUpdate(BaseModel):
"""部分更新:只改传入(非 None)字段。视频文件走 /video 上传接口。"""
enabled: bool | None = None
max_plays: int | None = Field(default=None, ge=0, le=MAX_PLAYS_LIMIT)
reward_coin: int | None = Field(default=None, ge=0, le=REWARD_COIN_LIMIT)
+36
View File
@@ -56,6 +56,42 @@ class PriceReportRejectRequest(BaseModel):
return v.strip()
class PriceReportBulkRequest(BaseModel):
ids: list[int] = Field(min_length=1, max_length=50, description="待审核上报 ID 列表")
@field_validator("ids")
@classmethod
def _ids_must_be_unique(cls, ids: list[int]) -> list[int]:
if len(ids) != len(set(ids)):
raise ValueError("上报 ID 不能重复")
return ids
class PriceReportBulkRejectRequest(PriceReportBulkRequest):
reason: str = Field(min_length=1, max_length=256, description="批量拒绝理由,用户端记录页会看到")
@field_validator("reason")
@classmethod
def _reason_not_blank(cls, value: str) -> str:
if not value.strip():
raise ValueError("拒绝理由不能为空")
return value.strip()
class PriceReportBulkItemResult(BaseModel):
id: int
ok: bool
status: str | None = None
error: str | None = None
class PriceReportBulkResult(BaseModel):
total: int
success: int
failed: int
items: list[PriceReportBulkItemResult]
class PriceReportSummary(BaseModel):
"""审核台顶部各状态计数。"""
+104
View File
@@ -0,0 +1,104 @@
"""风控监控后台接口 schema。"""
from __future__ import annotations
from datetime import datetime
from pydantic import BaseModel, Field
class RiskSummaryCard(BaseModel):
kind: str
alert_subject_count: int
today_total: int
threshold: int
rule_text: str
class RiskMonitorSummary(BaseModel):
date: str
updated_at: datetime
cards: list[RiskSummaryCard]
class RiskRuleConfig(BaseModel):
sms_hourly_threshold: int = Field(ge=1, le=5)
oneclick_daily_threshold: int = Field(ge=1, le=100_000)
compare_daily_threshold: int = Field(ge=1, le=100)
class RiskIncidentItem(BaseModel):
incident_id: int
kind: str
subject_type: str
subject_id: str
device_model: str | None = None
first_used_at: datetime | None = None
phone: str | None = None
user_id: int | None = None
registered_at: datetime | None = None
common_device_id: str | None = None
window_start: datetime
window_end: datetime
first_event_at: datetime
triggered_at: datetime
last_event_at: datetime
event_count: int
status: str
restricted: bool = False
restriction_id: int | None = None
class RiskIncidentPage(BaseModel):
items: list[RiskIncidentItem]
next_cursor: int | None = None
total: int
class RiskDetailItem(BaseModel):
id: int
occurred_at: datetime
outcome: str
phone: str | None = None
user_id: int | None = None
username: str | None = None
account_registered_at: datetime | None = None
verified: bool | None = None
reason: str | None = None
store_name: str | None = None
product_names: str | None = None
prices: dict[str, float | None] | None = None
saved_amount_yuan: float | None = None
status: str | None = None
class RiskDetailPage(BaseModel):
kind: str
incident_id: int
event_count: int
distinct_accounts: int | None = None
items: list[RiskDetailItem]
next_cursor: int | None = None
total: int
class RiskActionRequest(BaseModel):
reason: str = Field("", max_length=256)
class RiskActionResponse(BaseModel):
ok: bool = True
incident_id: int
status: str
restriction_id: int | None = None
class RiskResetResponse(BaseModel):
ok: bool = True
reset_at: datetime
reset_incident_count: int
reset_counts: dict[str, int]
class RestrictionRevokeResponse(BaseModel):
ok: bool = True
restriction_id: int
+19 -1
View File
@@ -16,6 +16,8 @@ class AdminUserListItem(BaseModel):
register_channel: str
status: str
debug_trace_enabled: bool = False
is_high_risk: bool = False
high_risk_note: str | None = None
wechat_openid: str | None = None
wechat_nickname: str | None = None
created_at: datetime
@@ -58,7 +60,7 @@ class UserRewardStats(BaseModel):
reward_video_avg_ecpm: float # 平均激励视频 eCPM(分/千次)
reward_video_cash_cents: int # 激励视频提现(金币折现)
feed_count: int # 累计信息流广告数(granted 份数,unit_count 累加)
feed_avg_ecpm: float # 平均信息流广告 eCPM(分/千次)
feed_avg_ecpm: float # 全部 Draw/feed 实际展示的平均 eCPM(分/千次,含未发奖展示)
feed_cash_cents: int # 信息流广告提现(金币折现)
@@ -118,3 +120,19 @@ class SetUserStatusRequest(BaseModel):
class SetDebugTraceRequest(BaseModel):
enabled: bool = Field(..., description="是否给该用户开「复制调试链接」权限")
class SetUserRiskRequest(BaseModel):
is_high_risk: bool = Field(..., description="是否标记为高风险用户")
note: str | None = Field(
None,
max_length=500,
description="高风险备注;标记高风险时必填,解除后清空",
)
@field_validator("note")
@classmethod
def validate_note(cls, value: str | None, info):
if info.data.get("is_high_risk") and not (value or "").strip():
raise ValueError("标记高风险时必须填写原因")
return value.strip() if value and value.strip() else None
+27 -25
View File
@@ -60,6 +60,8 @@ class WithdrawListItemOut(WithdrawOrderOut):
phone: str | None = None
nickname: str | None = None
is_high_risk: bool = False
high_risk_note: str | None = None
cumulative_success_cents: int = 0 # 累计成功提现 = SUM(amount_cents) WHERE status='success'
@@ -67,7 +69,10 @@ class WithdrawSummaryOut(BaseModel):
reviewing_count: int
reviewing_amount_cents: int
pending_count: int
success_count: int
rejected_count: int
failed_count: int
total_count: int
today_success_count: int
today_success_amount_cents: int
today_rejected_count: int
@@ -77,6 +82,8 @@ class WithdrawUserSnapshot(BaseModel):
id: int
phone: str
nickname: str | None = None
is_high_risk: bool = False
high_risk_note: str | None = None
status: str
wechat_nickname: str | None = None
wechat_avatar_url: str | None = None
@@ -87,19 +94,34 @@ class WithdrawUserSnapshot(BaseModel):
withdraw_success_cents: int
class InviteeDetailOut(BaseModel):
user_id: int
phone: str
registered_at: datetime
invite_success: bool
first_compare_store: str | None = None
first_compare_products: str | None = None
first_order_store: str | None = None
first_order_products: str | None = None
first_order_amount_cents: int | None = None
class InviteOverviewOut(BaseModel):
invite_total: int
invite_success_total: int
items: list[InviteeDetailOut]
class WithdrawDetailOut(BaseModel):
order: WithdrawOrderOut
user: WithdrawUserSnapshot | None = None
cumulative_success_cents: int = 0
risk_flags: list[str]
risk_score: int
recent_withdraws: list[WithdrawOrderOut]
recent_cash_transactions: list[CashTxnOut]
audit_logs: list[AdminAuditLogOut]
class ReconcileResult(BaseModel):
checked: int
resolved: int
invite_overview: InviteOverviewOut | None = None
class WithdrawBulkRequest(BaseModel):
@@ -128,26 +150,6 @@ class WithdrawBulkResult(BaseModel):
items: list[WithdrawBulkItemResult]
class WithdrawLedgerCheckOut(BaseModel):
ok: bool
# 普通现金账(coin_cash:金币兑换的现金)
cash_balance_total_cents: int
cash_transaction_total_cents: int
balance_diff_cents: int
missing_withdraw_txn_count: int
missing_refund_txn_count: int
duplicate_refund_txn_count: int
refund_txn_on_non_terminal_count: int
# 邀请奖励金账(invite_cash:与普通现金物理隔离,各自对账)。默认 0 向后兼容。
invite_cash_balance_total_cents: int = 0
invite_cash_transaction_total_cents: int = 0
invite_balance_diff_cents: int = 0
invite_missing_withdraw_txn_count: int = 0
invite_missing_refund_txn_count: int = 0
invite_duplicate_refund_txn_count: int = 0
invite_refund_txn_on_non_terminal_count: int = 0
class WxpayHealthCheckOut(BaseModel):
ok: bool
wxpay_configured: bool
+13 -1
View File
@@ -4,7 +4,7 @@ from __future__ import annotations
import logging
from typing import Annotated
from fastapi import Depends, HTTPException, status
from fastapi import Depends, HTTPException, Request, status
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
from sqlalchemy.orm import Session
@@ -77,6 +77,18 @@ def get_current_user_optional(
return user
def get_client_ip(request: Request) -> str:
"""客户端真实 IP:生产经 nginx 反代优先 X-Forwarded-For 首段,否则直连 IP。
路由层取 IP 的规范实现(analytics / applog 等共用);core 层(ratelimit)因不能
反向依赖 app.api,自留一份私有副本。
"""
xff = request.headers.get("x-forwarded-for")
if xff:
return xff.split(",")[0].strip()
return request.client.host if request.client else ""
CurrentUser = Annotated[User, Depends(get_current_user)]
OptionalUser = Annotated[User | None, Depends(get_current_user_optional)]
DbSession = Annotated[Session, Depends(get_db)]
+2 -10
View File
@@ -10,7 +10,7 @@ import logging
from fastapi import APIRouter, HTTPException, Request
from app.api.deps import DbSession
from app.api.deps import DbSession, get_client_ip
from app.repositories import analytics as analytics_repo
from app.repositories import analytics_selfstat as selfstat_repo
from app.schemas.analytics import AnalyticsBatchIn, AnalyticsIngestOut
@@ -20,19 +20,11 @@ router = APIRouter(prefix="/api/v1/analytics", tags=["analytics"])
logger = logging.getLogger("shagua.analytics")
def _client_ip(request: Request) -> str:
"""取客户端 IP:生产经 nginx 反代优先 X-Forwarded-For 第一段,否则直连 IP(同 admin get_client_ip)。"""
xff = request.headers.get("x-forwarded-for")
if xff:
return xff.split(",")[0].strip()
return request.client.host if request.client else ""
@router.post("/events", response_model=AnalyticsIngestOut, summary="批量上报埋点事件")
def ingest_events(
batch: AnalyticsBatchIn, request: Request, db: DbSession
) -> AnalyticsIngestOut:
n = analytics_repo.record_batch(db, batch, client_ip=_client_ip(request))
n = analytics_repo.record_batch(db, batch, client_ip=get_client_ip(request))
return AnalyticsIngestOut(received=n)
+50
View File
@@ -0,0 +1,50 @@
"""客户端运行日志批量上报接口。
POST /api/v1/applog/batch — 批量接收客户端运行日志,逐条写专用滚动文件 logs/app-client.log
(供 Logtail 采进独立 SLS logstore)。鉴权同 analytics(不强制登录,user_id 可选在 body)。
fire-and-forget:写失败也不 500(避免客户端重试风暴);超批 422、超体积 413、msg 超限截断。
"""
from __future__ import annotations
import os
from fastapi import APIRouter, Depends, HTTPException, Request
from app.api.deps import get_client_ip
from app.core.client_log import write_records
from app.core.ratelimit import rate_limit
from app.schemas.applog import AppLogBatchIn, AppLogIngestOut
router = APIRouter(prefix="/api/v1/applog", tags=["applog"])
def _enforce_body_limit(request: Request) -> None:
"""依赖:body 声明过大直接 413(在 body 校验前拦截)。缺 Content-Length 由 nginx 兜底。"""
max_bytes = int(os.getenv("APPLOG_MAX_BODY_BYTES", str(1024 * 1024 * 2)))
cl = request.headers.get("content-length")
if cl is not None and cl.isdigit() and int(cl) > max_bytes:
raise HTTPException(status_code=413, detail="日志批量过大")
@router.post(
"/batch",
response_model=AppLogIngestOut,
summary="批量上报客户端运行日志",
dependencies=[
Depends(rate_limit(120, 60, "applog-batch")),
Depends(_enforce_body_limit),
],
)
def ingest_logs(batch: AppLogBatchIn, request: Request) -> AppLogIngestOut:
received, dropped = write_records(
batch.logs,
meta={
"device_id": batch.device_id,
"user_id": batch.user_id,
"app_ver": batch.app_ver,
"platform": batch.platform,
"sent_at": batch.sent_at,
},
client_ip=get_client_ip(request),
)
return AppLogIngestOut(received=received, dropped=dropped)
+147 -4
View File
@@ -37,6 +37,7 @@ from app.integrations.jiguang import JiguangError, mask_phone, verify_and_get_ph
from app.integrations.sms import SmsError, send_code, verify_code
from app.repositories import onboarding as onboarding_repo
from app.repositories import phone_rebind as rebind_repo
from app.repositories import risk as risk_repo
from app.repositories import user as user_repo
from app.schemas.auth import (
JverifyLoginRequest,
@@ -70,6 +71,18 @@ SMS_SEND_MAX_PER_HOUR_PER_DEVICE = 5 # 每小时上限
SMS_SEND_MAX_PER_DAY_PER_DEVICE = 20 # 每天上限(再叠一层日封顶,挡低频长时间轰炸)
def _client_ip(request: Request) -> str:
xff = request.headers.get("x-forwarded-for")
if xff:
return xff.split(",")[0].strip()
return request.client.host if request.client else ""
def _device_subject(device_id: str, request: Request) -> str:
# 兼容旧客户端空 device_id;新客户端均上传稳定硬件标识。
return device_id or f"legacy-ip:{_client_ip(request)}"
def _login_response(
user, *, onboarding_completed: bool, force_onboarding: bool = False
) -> TokenWithUser:
@@ -87,7 +100,15 @@ def _login_response(
# ===================== 极光一键登录 =====================
@router.post("/jverify-login", response_model=TokenWithUser, summary="极光一键登录")
def jverify_login(req: JverifyLoginRequest, db: DbSession) -> TokenWithUser:
def jverify_login(req: JverifyLoginRequest, request: Request, db: DbSession) -> TokenWithUser:
subject_id = _device_subject(req.device_id, request)
if req.device_id and risk_repo.is_restricted(
db,
subject_type="device",
subject_id=req.device_id,
scope=risk_repo.SCOPE_AUTH_DEVICE,
):
raise HTTPException(status_code=403, detail="当前设备环境异常,暂无法登录")
logger.info(
"jverify_login operator=%s token_len=%d",
req.operator or "-",
@@ -97,10 +118,37 @@ def jverify_login(req: JverifyLoginRequest, db: DbSession) -> TokenWithUser:
try:
phone = verify_and_get_phone(req.login_token)
except JiguangError as e:
risk_repo.record_behavior_event(
db,
event_type=risk_repo.EVENT_ONECLICK_LOGIN,
subject_type="device",
subject_id=subject_id,
device_id=req.device_id or None,
device_model=req.device_model or None,
client_ip=_client_ip(request),
outcome="failed",
reason=str(e),
details={"operator": req.operator or None},
evaluate_rule=risk_repo.RULE_ONECLICK_DAILY,
)
logger.error("[JG] verify+decrypt failed: %s", e, exc_info=True)
raise HTTPException(status_code=502, detail=f"jiguang verify failed: {e}") from e
user = user_repo.upsert_user_for_login(db, phone=phone, register_channel="jverify")
risk_repo.record_behavior_event(
db,
event_type=risk_repo.EVENT_ONECLICK_LOGIN,
subject_type="device",
subject_id=subject_id,
user_id=user.id,
device_id=req.device_id or None,
device_model=req.device_model or None,
phone=phone,
client_ip=_client_ip(request),
outcome="success",
details={"operator": req.operator or None},
evaluate_rule=risk_repo.RULE_ONECLICK_DAILY,
)
if user.status != "active":
raise HTTPException(status_code=403, detail="account disabled")
@@ -116,12 +164,34 @@ def jverify_login(req: JverifyLoginRequest, db: DbSession) -> TokenWithUser:
response_model=SmsSendResponse,
summary="发送短信验证码",
)
def sms_send(req: SmsSendRequest, request: Request) -> SmsSendResponse:
def sms_send(req: SmsSendRequest, request: Request, db: DbSession) -> SmsSendResponse:
subject_id = _device_subject(req.device_id, request)
if req.device_id and risk_repo.is_restricted(
db,
subject_type="device",
subject_id=req.device_id,
scope=risk_repo.SCOPE_AUTH_DEVICE,
):
raise HTTPException(status_code=403, detail="当前设备环境异常,暂无法发送验证码")
# 测试账号:不真发短信(号码非真实手机号,真发会失败/浪费),直接放行让客户端进入填码界面。
# 真正的"免验证码"在 /sms/login 跳过校验;每日上限也在 login 处算(send 不耗额度)。
# (也不受下面设备发码限流约束:QA 联调要反复发码。)
if test_account.is_test_account(req.phone):
logger.info("test_account sms_send short-circuit (不真发)")
risk_repo.record_behavior_event(
db,
event_type=risk_repo.EVENT_SMS_SEND,
subject_type="device",
subject_id=subject_id,
device_id=req.device_id or None,
device_model=req.device_model or None,
phone=req.phone,
client_ip=_client_ip(request),
# 测试号不真发短信,也豁免正式频控;保留流水供排查,但不能污染
# “今日短信下发”与短信风控报警。
outcome="test",
details={"mock": True, "test_account": True},
)
return SmsSendResponse(sent=True, mock=True, cooldown_sec=0)
# 发码防刷:同一设备(device_id) + 同一 IP,每小时 / 每天两道闸,**均只按成功发码计数**。
@@ -139,6 +209,18 @@ def sms_send(req: SmsSendRequest, request: Request) -> SmsSendResponse:
try:
cooldown = send_code(req.phone)
except SmsError as e:
risk_repo.record_behavior_event(
db,
event_type=risk_repo.EVENT_SMS_SEND,
subject_type="device",
subject_id=subject_id,
device_id=req.device_id or None,
device_model=req.device_model or None,
phone=req.phone,
client_ip=_client_ip(request),
outcome="failed",
reason=str(e),
)
raise HTTPException(status_code=e.status_code, detail=str(e)) from e
# 发码成功 → 两道闸各 +1(被单号冷却挡下的重发走不到这里,故不占额度)
@@ -146,6 +228,20 @@ def sms_send(req: SmsSendRequest, request: Request) -> SmsSendResponse:
from app.core.config import settings # 局部 import 避免循环
risk_repo.record_behavior_event(
db,
event_type=risk_repo.EVENT_SMS_SEND,
subject_type="device",
subject_id=subject_id,
device_id=req.device_id or None,
device_model=req.device_model or None,
phone=req.phone,
client_ip=_client_ip(request),
outcome="success",
details={"mock": settings.SMS_MOCK},
evaluate_rule=risk_repo.RULE_SMS_HOURLY,
)
return SmsSendResponse(sent=True, mock=settings.SMS_MOCK, cooldown_sec=cooldown)
@@ -155,6 +251,7 @@ def sms_send(req: SmsSendRequest, request: Request) -> SmsSendResponse:
summary="手机号+验证码登录",
)
def sms_login(req: SmsLoginRequest, request: Request, db: DbSession) -> TokenWithUser:
subject_id = _device_subject(req.device_id, request)
# 测试账号:免验证码登录 + 每日上限 + 每次都走新手引导(详见 app/core/test_account.py)。
# 放在最前面:命中即不校验验证码;先扣当日额度,超限直接拒,挡住有人猜到号后脚本刷。
# 测试账号走自己的每日额度、不受下面 (设备+IP) 每小时限流约束(QA 需在一小时内反复登录联调)。
@@ -162,6 +259,19 @@ def sms_login(req: SmsLoginRequest, request: Request, db: DbSession) -> TokenWit
if not test_account.try_consume_quota():
raise HTTPException(status_code=429, detail="测试账号今日使用次数已达上限,请明天再试")
user = user_repo.upsert_user_for_login(db, phone=req.phone, register_channel="sms")
risk_repo.record_behavior_event(
db,
event_type=risk_repo.EVENT_SMS_LOGIN,
subject_type="device",
subject_id=subject_id,
user_id=user.id,
device_id=req.device_id or None,
device_model=req.device_model or None,
phone=req.phone,
client_ip=_client_ip(request),
outcome="success",
details={"test_account": True},
)
if user.status != "active":
raise HTTPException(status_code=403, detail="account disabled")
logger.info("sms_login test_account ok user_id=%d phone=%s", user.id, mask_phone(req.phone))
@@ -182,10 +292,39 @@ def sms_login(req: SmsLoginRequest, request: Request, db: DbSession) -> TokenWit
detail="登录尝试过于频繁,请稍后再试",
)
if not verify_code(req.phone, req.code):
try:
ok = verify_code(req.phone, req.code)
except SmsError as e: # provider 校验降级(如阿里云接口异常)→ 原样透出其状态码(503),别误报「验证码错误」
raise HTTPException(status_code=e.status_code, detail=str(e)) from e
if not ok:
# 校验码错误才记风控失败事件(provider 降级 503 已在上面提前 raise,不算「验证失败」)
risk_repo.record_behavior_event(
db,
event_type=risk_repo.EVENT_SMS_LOGIN,
subject_type="device",
subject_id=subject_id,
device_id=req.device_id or None,
device_model=req.device_model or None,
phone=req.phone,
client_ip=_client_ip(request),
outcome="failed",
reason="invalid sms code",
)
raise HTTPException(status_code=400, detail="invalid sms code")
user = user_repo.upsert_user_for_login(db, phone=req.phone, register_channel="sms")
risk_repo.record_behavior_event(
db,
event_type=risk_repo.EVENT_SMS_LOGIN,
subject_type="device",
subject_id=subject_id,
user_id=user.id,
device_id=req.device_id or None,
device_model=req.device_model or None,
phone=req.phone,
client_ip=_client_ip(request),
outcome="success",
)
if user.status != "active":
raise HTTPException(status_code=403, detail="account disabled")
@@ -322,7 +461,11 @@ def wechat_bind_phone_sms(
detail="登录尝试过于频繁,请稍后再试",
)
if not verify_code(req.phone, req.code):
try:
ok = verify_code(req.phone, req.code)
except SmsError as e: # provider 校验降级(如阿里云接口异常)→ 原样透出其状态码(503),别误报「验证码错误」
raise HTTPException(status_code=e.status_code, detail=str(e)) from e
if not ok:
raise HTTPException(status_code=400, detail="invalid sms code")
return _finish_wechat_bind(
+64 -13
View File
@@ -25,22 +25,34 @@ import uuid
from typing import Any
import httpx
from fastapi import APIRouter, HTTPException, Request, status
from fastapi import APIRouter, BackgroundTasks, HTTPException, Request, status
from fastapi.concurrency import run_in_threadpool
from app.api.deps import OptionalUser
from app.api.deps import DbSession, OptionalUser
from app.core.config import settings
from app.core.logging import trace_id_ctx
from app.core.pricebot_client import get_pricebot_client
from app.core.pricebot_router import pick_pricebot
from app.db.session import SessionLocal
from app.repositories import comparison as crud_compare
from app.repositories import risk as risk_repo
from app.services.comparison_llm_backfill import backfill_comparison_llm_cost
logger = logging.getLogger("shagua.compare")
router = APIRouter(prefix="/api/v1", tags=["compare"])
def _ensure_compare_allowed(user, db) -> None:
if user and risk_repo.is_restricted(
db,
subject_type="user",
subject_id=str(user.id),
scope=risk_repo.SCOPE_ECONOMIC_ACCOUNT,
):
raise HTTPException(status_code=403, detail="账号存在异常,该功能暂不可用")
# ============================================================
# harvest 落库(阻塞 SQLAlchemy → run_in_threadpool,独立 SessionLocal,不阻塞事件循环;
# 任何写库异常都吞掉、绝不连累比价透传返回 —— 同 coupon.py 现有 best-effort 写法)。
@@ -52,10 +64,14 @@ def _harvest_running_blocking(
device_id: str | None, device_info: dict | None, trace_url: str | None,
) -> None:
with SessionLocal() as db:
crud_compare.harvest_running(
rec = crud_compare.harvest_running(
db, trace_id=trace_id, user_id=user_id, business_type=business_type,
device_id=device_id, device_info=device_info, trace_url=trace_url,
)
if rec.user_id is not None:
risk_repo.sync_compare_incident(
db, user_id=rec.user_id, at=rec.created_at
)
logger.info(
"harvest running row (user=%s)", user_id,
extra={"phase": "harvest_running", "status": "running", "user_id": user_id},
@@ -65,13 +81,17 @@ def _harvest_running_blocking(
def _harvest_done_blocking(
trace_id: str, user_id: int | None, done_params: dict, business_type: str,
device_id: str | None, device_info: dict | None, trace_url: str | None,
) -> None:
) -> int:
with SessionLocal() as db:
rec, newly_success = crud_compare.harvest_done(
db, trace_id=trace_id, user_id=user_id, done_params=done_params,
business_type=business_type, device_id=device_id,
device_info=device_info, trace_url=trace_url,
)
if rec.user_id is not None:
risk_repo.sync_compare_incident(
db, user_id=rec.user_id, at=rec.created_at
)
logger.info(
"harvest done → %s saved=%s newly=%s", rec.status,
rec.saved_amount_cents, newly_success,
@@ -83,11 +103,12 @@ def _harvest_done_blocking(
# 不在此处发邀请奖:#113 已把发奖口径从「比价」移到「实际下单」(order.py),harvest
# 只记录比价、不发奖。否则比价先于下单 + try_reward 幂等闸会让奖落在「比价」这步,
# 架空 #113 的「下单才发奖」防刷意图(newly_success 仅留作日志观测)。
return rec.id
def _harvest_abort_blocking(
trace_id: str, status_hint: str, reason: str | None, trace_url: str | None,
) -> None:
) -> int | None:
with SessionLocal() as db:
rec = crud_compare.harvest_abort(
db, trace_id=trace_id, status=status_hint, reason=reason, trace_url=trace_url,
@@ -97,6 +118,7 @@ def _harvest_abort_blocking(
extra={"phase": "harvest_abort",
"status": (rec.status if rec else None), "reason": reason},
)
return rec.id if rec is not None else None
async def _forward(
@@ -199,25 +221,38 @@ async def _forward(
@router.post("/intent/recognize", summary="外卖比价 Phase 1 意图识别 (透传 + 建 running 行)")
async def intent_recognize(request: Request, user: OptionalUser) -> dict[str, Any]:
async def intent_recognize(
request: Request, user: OptionalUser, db: DbSession
) -> dict[str, Any]:
_ensure_compare_allowed(user, db)
resp, _, _ = await _forward(request, "/api/intent/recognize", user)
return resp
@router.post("/intent/step", summary="外卖比价 Phase 1 多帧意图识别 (透传, 仅淘宝源)")
async def intent_step(request: Request, user: OptionalUser) -> dict[str, Any]:
async def intent_step(request: Request, user: OptionalUser, db: DbSession) -> dict[str, Any]:
_ensure_compare_allowed(user, db)
resp, _, _ = await _forward(request, "/api/intent/step", user)
return resp
@router.post("/intent/precoupon/step", summary="外卖比价 Phase 0 识别前先用券 (透传, 仅美团源)")
async def intent_precoupon_step(request: Request, user: OptionalUser) -> dict[str, Any]:
async def intent_precoupon_step(
request: Request, user: OptionalUser, db: DbSession
) -> dict[str, Any]:
_ensure_compare_allowed(user, db)
resp, _, _ = await _forward(request, "/api/intent/precoupon/step", user)
return resp
@router.post("/price/step", summary="外卖比价 Phase 2 步进 (透传 + done 落库)")
async def price_step(request: Request, user: OptionalUser) -> dict[str, Any]:
async def price_step(
request: Request,
background_tasks: BackgroundTasks,
user: OptionalUser,
db: DbSession,
) -> dict[str, Any]:
_ensure_compare_allowed(user, db)
resp, trace_id, meta = await _forward(request, "/api/price/step", user)
# 最终 done 帧(command=done 且 continue=false)→ harvest 更新成终态。
# (单平台中途 done 被 pricebot 改写成 wait+continue=true,不会命中这里,同 coupon 语义。)
@@ -225,19 +260,25 @@ async def price_step(request: Request, user: OptionalUser) -> dict[str, Any]:
if action.get("command") == "done" and not resp.get("continue", True):
done_params = action.get("params") or {}
try:
await run_in_threadpool(
record_id = await run_in_threadpool(
_harvest_done_blocking, trace_id, (user.id if user else None),
done_params, "food",
meta.get("device_id"), meta.get("device_info"),
resp.get("trace_url") or done_params.get("trace_url"),
)
background_tasks.add_task(
backfill_comparison_llm_cost, record_id, trace_id
)
except Exception as e: # noqa: BLE001
logger.warning("harvest_done failed trace=%s: %s", trace_id, e)
return resp
@router.post("/trace/epilogue", summary="比价结果页尾声帧 (透传到 pricebot, 用户视角截图入 trace)")
async def trace_epilogue(request: Request, user: OptionalUser) -> dict[str, Any]:
async def trace_epilogue(
request: Request, user: OptionalUser, db: DbSession
) -> dict[str, Any]:
_ensure_compare_allowed(user, db)
# App 收到 done、渲染完结果页后,把自己页面的截图(base64)传给 pricebot 存进 trace
# 目录并触发重传 —— trace 里补上"用户实际看到的汇总页"(步骤帧只有目标 App 画面)。
# body ~几百 KB(截图 base64), 纯透传壳(harvest_first_frame=False:不建行/不落库,
@@ -250,7 +291,13 @@ async def trace_epilogue(request: Request, user: OptionalUser) -> dict[str, Any]
@router.post("/trace/finalize", summary="比价 trace 收尾上云 (透传 + 夭折落库)")
async def trace_finalize(request: Request, user: OptionalUser) -> dict[str, Any]:
async def trace_finalize(
request: Request,
background_tasks: BackgroundTasks,
user: OptionalUser,
db: DbSession,
) -> dict[str, Any]:
_ensure_compare_allowed(user, db)
# 用户终止 / Phase1 未识别没到 done 帧: pricebot 打包半截上云返回 {trace_url};
# app-server 顺手把该 trace 的 running 行更新成夭折终态(**不降级 success**)。
# 客户端 finalize body 带 status(cancelled/failed)+ reason;老客户端只带 trace_id →
@@ -259,12 +306,16 @@ async def trace_finalize(request: Request, user: OptionalUser) -> dict[str, Any]
request, "/api/trace/finalize", user, harvest_first_frame=False,
)
try:
await run_in_threadpool(
record_id = await run_in_threadpool(
_harvest_abort_blocking, trace_id,
(meta.get("status") or "cancelled"),
(meta.get("reason") or meta.get("information")),
(resp.get("trace_url") if isinstance(resp, dict) else None),
)
if record_id is not None:
background_tasks.add_task(
backfill_comparison_llm_cost, record_id, trace_id
)
except Exception as e: # noqa: BLE001
logger.warning("harvest_abort failed trace=%s: %s", trace_id, e)
return resp
+56 -30
View File
@@ -16,10 +16,11 @@ import logging
from fastapi import APIRouter, BackgroundTasks, HTTPException, Query, status
from app.api.deps import CurrentUser, DbSession
from app.db.session import SessionLocal
from app.models.comparison import ComparisonRecord
from app.repositories import comparison as crud_compare
from app.repositories import risk as risk_repo
from app.schemas.compare_record import (
CompareStartReserveIn,
CompareStartReserveOut,
CompareStatsOut,
ComparisonRecordCreatedOut,
ComparisonRecordDetailOut,
@@ -27,14 +28,57 @@ from app.schemas.compare_record import (
ComparisonRecordOut,
ComparisonRecordPage,
)
from app.services.llm_cost import compute_llm_cost, get_llm_prices
from app.services.pricebot_llm_calls import fetch_llm_calls
from app.services.comparison_llm_backfill import backfill_comparison_llm_cost
logger = logging.getLogger("shagua.compare_record")
router = APIRouter(prefix="/api/v1/compare", tags=["compare-record"])
@router.post(
"/start",
response_model=CompareStartReserveOut,
summary="预占一次当日比价发起次数(每人每天最多100次)",
)
def reserve_compare_start(
payload: CompareStartReserveIn,
user: CurrentUser,
db: DbSession,
) -> CompareStartReserveOut:
if risk_repo.is_restricted(
db,
subject_type="user",
subject_id=str(user.id),
scope=risk_repo.SCOPE_ECONOMIC_ACCOUNT,
):
raise HTTPException(status_code=403, detail="账号存在异常,该功能暂不可用")
try:
rec, used = crud_compare.reserve_daily_start(
db,
user_id=user.id,
trace_id=payload.trace_id,
business_type=payload.business_type,
device_id=payload.device_id,
)
except crud_compare.DailyCompareStartLimitExceeded:
raise HTTPException(
status_code=status.HTTP_429_TOO_MANY_REQUESTS,
detail="今日已比价超过100次,请明天再试",
) from None
except crud_compare.ComparisonTraceOwnershipError:
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail="比价任务标识冲突,请重新发起",
) from None
# 风控阈值由后台动态配置,不能再只在固定的 100 次业务上限处同步。
risk_repo.sync_compare_incident(db, user_id=user.id, at=rec.created_at)
return CompareStartReserveOut(
limit=crud_compare.DAILY_COMPARE_START_LIMIT,
used=used,
remaining=max(crud_compare.DAILY_COMPARE_START_LIMIT - used, 0),
)
@router.post(
"/record",
response_model=ComparisonRecordCreatedOut,
@@ -46,6 +90,13 @@ def report_record(
db: DbSession,
background_tasks: BackgroundTasks,
) -> ComparisonRecordCreatedOut:
if risk_repo.is_restricted(
db,
subject_type="user",
subject_id=str(user.id),
scope=risk_repo.SCOPE_ECONOMIC_ACCOUNT,
):
raise HTTPException(status_code=403, detail="账号存在异常,该功能暂不可用")
rec = crud_compare.upsert_record(db, user_id=user.id, payload=payload)
# LLM 调用明细异步回填:同机拉 pricebot llm_calls 最长 5s 且是 best-effort,放后台
# 任务做,不阻塞上报响应(顺带给 pricebot 落盘留足余量)。upsert 已 commit,后台用
@@ -67,32 +118,7 @@ def report_record(
def _backfill_llm_calls(record_id: int, trace_id: str) -> None:
"""后台回填本次比价的 LLM 调用明细 + 派生 llm_call_count/retry_count。
独立 DB session(请求 session 此时已关);拉取/写库失败只 log,绝不影响已落库的上报"""
calls = fetch_llm_calls(trace_id)
if not calls:
return
db = SessionLocal()
try:
rec = db.get(ComparisonRecord, record_id)
if rec is None:
return
rec.llm_calls = calls
rec.llm_call_count = len(calls)
rec.retry_count = sum(1 for c in calls if c.get("error"))
# token 累加(usage 已被 pricebot llm_client 归一为 prompt/completion_tokens;
# error 的调用 usage 可能为 None,or {} 兜底)
rec.input_tokens = sum((c.get("usage") or {}).get("prompt_tokens") or 0 for c in calls)
rec.output_tokens = sum((c.get("usage") or {}).get("completion_tokens") or 0 for c in calls)
# 本次比价 LLM 成本(元)+ 当时单价快照:按 app_config 现价逐模型算好冻结(services/llm_cost.py)。
rec.llm_cost_yuan, rec.llm_price_snapshot = compute_llm_cost(calls, get_llm_prices(db))
db.commit()
logger.info(
"backfill llm_calls trace=%s n=%d in_tok=%d out_tok=%d",
trace_id, len(calls), rec.input_tokens, rec.output_tokens,
)
except Exception as e: # noqa: BLE001 best-effort
logger.warning("backfill llm_calls failed trace=%s: %s", trace_id, e)
finally:
db.close()
backfill_comparison_llm_cost(record_id, trace_id)
@router.get(
+2 -2
View File
@@ -81,7 +81,7 @@ def _record_claims_blocking(
device_id: str, user_id: int | None, trace_id: str | None, results: list[dict]
) -> None:
with SessionLocal() as db:
# 取本次 session 环境,给 coupon_claim_record 打 app_env 标(每券成功率表按它过滤;设计 §13)
# 取本次 session 环境,给每日资产和逐次事件同时打环境标
app_env = coupon_repo.session_app_env(db, trace_id)
coupon_repo.record_claims(db, device_id, user_id, trace_id, results, app_env=app_env)
# 顺带把本帧「成功平台」并入 coupon_session.platform_success(admin 领券数据 ②整单/③点位成功率;
@@ -176,7 +176,7 @@ async def coupon_step(
resp_json = resp.json()
# 领券结果沉淀(资产):中间帧 last_coupon_result + done 帧 coupon_results 幂等写库。
# 领券结果沉淀:每日资产 + 逐次事件;中间帧和 done 全量帧均幂等写库。
# 当前只记录、不参与"要不要领"判断(MVP 先不去重)。写库失败不影响返回。
if device_id:
results = _extract_coupon_results(resp_json)
+64
View File
@@ -0,0 +1,64 @@
"""新手引导视频(领券等候浮层前 N 次替代广告)。
路由前缀 `/api/v1/guide-video`(均需 Bearer):
POST /start 这次浮层放引导视频还是放广告?命中则**当场计次**并下发 play_token
POST /reward 播完 / 中途关闭都调, play_token 幂等发固定金币
发币额度以**服务端配置**为准(运营后台可改),客户端只报"播完/关闭",报不了金额,
所以被破解也刷不到超额金币;次数上限由 guide_video_play 行数(按账号)硬卡
"""
from __future__ import annotations
import logging
from fastapi import APIRouter, Depends
from app.api.deps import CurrentUser, DbSession
from app.core.ratelimit import rate_limit
from app.repositories import guide_video as crud_guide
from app.schemas.guide_video import (
GuideVideoRewardIn,
GuideVideoRewardOut,
GuideVideoStartIn,
GuideVideoStartOut,
)
logger = logging.getLogger("shagua.guide_video")
router = APIRouter(prefix="/api/v1/guide-video", tags=["guide-video"])
@router.post(
"/start",
response_model=GuideVideoStartOut,
summary="领券浮层是否放新手引导视频(命中即计次)",
dependencies=[Depends(rate_limit(60, 60, "guide-video-start"))],
)
def start(payload: GuideVideoStartIn, user: CurrentUser, db: DbSession) -> GuideVideoStartOut:
"""开播即计数:返回 should_play=True 时服务端已写下这一次,客户端必须真的播。
没配视频 / 开关关 / 次数用完 should_play=False,客户端照旧走广告链路(行为不变)
"""
result = crud_guide.start_play(db, user.id, scene=payload.scene or "coupon")
logger.info(
"guide video start user_id=%d scene=%s should_play=%s seq=%d remaining=%d",
user.id, payload.scene, result["should_play"], result["seq"], result["remaining"],
)
return GuideVideoStartOut(**result)
@router.post(
"/reward",
response_model=GuideVideoRewardOut,
summary="引导视频发金币(播完/中途关闭都发,play_token 幂等)",
dependencies=[Depends(rate_limit(60, 60, "guide-video-reward"))],
)
def reward(payload: GuideVideoRewardIn, user: CurrentUser, db: DbSession) -> GuideVideoRewardOut:
result = crud_guide.grant_play(
db, user.id, play_token=payload.play_token, completed=payload.completed
)
logger.info(
"guide video reward user_id=%d token=%s completed=%s granted=%s coin=%d",
user.id, payload.play_token[:12], payload.completed, result["granted"], result["coin"],
)
return GuideVideoRewardOut(**result)
+154 -75
View File
@@ -5,11 +5,12 @@
from __future__ import annotations
import logging
from concurrent.futures import ThreadPoolExecutor, as_completed
from concurrent.futures import ThreadPoolExecutor
from typing import TYPE_CHECKING, Any
from fastapi import APIRouter, Depends, HTTPException
from sqlalchemy import nullslast, select
from sqlalchemy.orm import Session, aliased
from sqlalchemy.orm import Session
from app.core.config import settings
from app.db.session import get_db
@@ -25,8 +26,14 @@ from app.schemas.meituan import (
ReferralLinkResponse,
TopSalesRequest,
)
from app.utils import mt_search_cursor
from app.utils.meituan_city import get_meituan_city
if TYPE_CHECKING: # 仅供类型标注(本模块已开 from __future__ import annotations)
from collections.abc import Callable
from sqlalchemy import ColumnElement
logger = logging.getLogger("shagua.meituan")
@@ -109,6 +116,86 @@ def _commission_pct(card: CouponCard) -> float:
return 0.0
# ────────────── 离线库分页(智能推荐 / 销量最高 共用) ──────────────
# 去重+排序阶段**只投影这几列**:够 DISTINCT ON 分组、够排序、够回表定位,且全是定长小字段。
# ⚠️ 关键性能点:`raw` 是整条美团原始返回(JSONB,每行数 KB)。原实现用 select(MeituanCoupon)
# 做子查询,等于把整城几千行连 raw 一起塞进两次排序(DISTINCT ON 一次 + 分页一次),
# 体量轻松超过 work_mem → Postgres 落盘做外部归并排序,而且**每翻一页都要重来一遍**。
# 拆成「先在小列上排出本页 id,再按 id 回表取 raw」后,排序数据量降到原来的百分之几,
# JSONB 只解析当前页 ~20 行。
_DEDUP_COLS = (
MeituanCoupon.id,
MeituanCoupon.dedup_key,
MeituanCoupon.sale_volume_num,
MeituanCoupon.commission_percent,
)
def _paged_dedup_ids(
db: Session,
*,
conds: list[ColumnElement[bool]],
dedup_order: list[ColumnElement],
page_order: Callable[[Any], list[ColumnElement]],
page: int,
page_size: int,
) -> tuple[list[int], bool]:
"""DISTINCT ON(dedup_key) 跨源去重 → 整体排序 → 分页,返回 (本页 id 列表, 是否还有下一页)。
- `dedup_order`:同一个 dedup_key 的多条里留哪条(如销量最高/佣金最高)
- `page_order`:接收去重子查询的列集合(`sub.c`),返回去重后的整体排序
多取 1 条用于判断 has_next
"""
deduped = (
select(*_DEDUP_COLS)
.where(*conds)
.distinct(MeituanCoupon.dedup_key)
.order_by(MeituanCoupon.dedup_key, *dedup_order)
.subquery()
)
ids = db.execute(
select(deduped.c.id)
.order_by(*page_order(deduped.c))
.offset((page - 1) * page_size)
.limit(page_size + 1)
).scalars().all()
return list(ids[:page_size]), len(ids) > page_size
def _load_raws(db: Session, ids: list[int]) -> list[dict]:
"""按给定 id 顺序取 raw(只回表本页 ~20 行)。缺行(被 ETL 清掉)静默跳过。"""
if not ids:
return []
raw_by_id = {
row_id: raw
for row_id, raw in db.execute(
select(MeituanCoupon.id, MeituanCoupon.raw).where(MeituanCoupon.id.in_(ids))
).all()
}
return [raw_by_id[i] for i in ids if i in raw_by_id]
def _cards_from_raws(raws: list[dict], *, hide_distance: bool) -> list[CouponCard]:
"""raw → CouponCard;解析失败的单条跳过,不整页失败。
hide_distance:离线库里的距离是相对城市默认点算的,对用户无意义且误导 智能推荐 /
销量最高两个 tab 一律置空,前端距离 店名那行只剩店名自动顶到最左
"""
cards: list[CouponCard] = []
for raw in raws:
try:
card = CouponCard.from_raw(raw or {})
except Exception: # noqa: BLE001
continue
if not card.product_view_sign:
continue
if hide_distance:
card.distance_text = None
card.distance_meters = None
cards.append(card)
return cards
@router.post("/feed", response_model=FeedResponse, summary="混合feed(外卖+到店交叉);tab=rec智能推荐/distance距离最近")
def feed(req: FeedRequest, db: Session = Depends(get_db)) -> FeedResponse:
lon, lat = req.longitude, req.latitude
@@ -133,17 +220,21 @@ def feed(req: FeedRequest, db: Session = Depends(get_db)) -> FeedResponse:
return [], True
# 距离最近:搜索召回(外卖搜"外卖" + 到店搜"美食",都 sortField=6 离我最近)一页页拉。
# 搜索翻页必须用 searchId(pageNo 翻不动),所以每个 feed 页顺序翻到第 N 页;两路并行、page 1 最快。
# 无状态、不改 APP(传页码即可);按你位置实时算距离(库里没存 POI 经纬度,只能实时)
# 搜索翻页必须用 searchId(pageNo 翻不动),而接口是无状态的(客户端只传页码)—— 原实现因此
# 每次都从第 1 页顺序重放到第 N 页,取第 N 页要向美团发 N 次请求,越往下滑越慢
# 现在把沿途 searchId 记进 [mt_search_cursor],稳态下每翻一页恒定 1 次请求;两路仍并行。
# 按你位置实时算距离(库里没存 POI 经纬度,只能实时)。
if tab == "distance":
lon_i, lat_i = int(lon * 1_000_000), int(lat * 1_000_000)
def _search_page_n(platform: int, biz_line: int | None, keyword: str, n: int) -> tuple[list[dict], bool, bool]:
"""顺序翻到第 n 页(搜索须 searchId 续页),返回(第 n 页 items, 是否还有下一页, 是否调用失败)。"""
sid: str | None = None
def _replay(
platform: int, biz_line: int | None, keyword: str,
key: mt_search_cursor.RouteKey, start: int, sid: str | None, n: int,
) -> tuple[list[dict], bool, bool]:
"""从第 start 页(用 sid 取)顺序翻到第 n 页。start==1 时 sid 应为 None(走 pageNo=1)。"""
data: list[dict] = []
has_next = False
for pg in range(1, n + 1):
for pg in range(start, n + 1):
body: dict = {
"platform": platform, "searchText": keyword, "sortField": 6,
"longitude": lon_i, "latitude": lat_i, "pageSize": 20,
@@ -161,10 +252,27 @@ def feed(req: FeedRequest, db: Session = Depends(get_db)) -> FeedResponse:
data = r.get("data") or []
sid = r.get("searchId")
has_next = bool(r.get("hasNext")) and bool(data)
# 记下「下一页要用哪个 searchId」;没有下一页就别记,免得存进死游标。
if sid and has_next:
mt_search_cursor.remember(key, pg + 1, sid)
if not data or (not has_next and pg < n):
return [], False, False # 没那么多页了(非错误)
return data, has_next, False
def _search_page_n(platform: int, biz_line: int | None, keyword: str, n: int) -> tuple[list[dict], bool, bool]:
"""取第 n 页,返回(第 n 页 items, 是否还有下一页, 是否调用失败)。
优先用缓存游标一发直达;缓存未命中/过期才从最近的已知页往后重放,并把沿途游标补进缓存
"""
key = mt_search_cursor.route_key(lat, lon, platform, keyword)
start, sid = mt_search_cursor.lookup(key, n)
data, has_next, failed = _replay(platform, biz_line, keyword, key, start, sid, n)
# 用缓存游标却打不通,多半是上游 searchId 过期:作废整条路线,回到第 1 页重放一次。
if failed and start > 1:
mt_search_cursor.drop(key)
data, has_next, failed = _replay(platform, biz_line, keyword, key, 1, None, n)
return data, has_next, failed
with ThreadPoolExecutor(max_workers=2) as pool:
f_wm = pool.submit(_search_page_n, 1, None, "外卖", req.page)
f_dd = pool.submit(_search_page_n, 2, 1, "美食", req.page)
@@ -194,39 +302,25 @@ def feed(req: FeedRequest, db: Session = Depends(get_db)) -> FeedResponse:
return FeedResponse(items=[], has_next=False, page=req.page, status="degraded")
PAGE = 20
try:
base = select(MeituanCoupon).where(
MeituanCoupon.commission_percent >= 3.0,
MeituanCoupon.city_id == city_id,
)
deduped = base.distinct(MeituanCoupon.dedup_key).order_by(
MeituanCoupon.dedup_key,
MeituanCoupon.commission_percent.desc(),
).subquery()
m = aliased(MeituanCoupon, deduped)
start = (req.page - 1) * PAGE
rows = db.execute(
select(m)
ids, has_next = _paged_dedup_ids(
db,
conds=[
MeituanCoupon.commission_percent >= 3.0,
MeituanCoupon.city_id == city_id,
],
# 同一去重键留佣金最高那条
dedup_order=[MeituanCoupon.commission_percent.desc()],
# 销量高的优先(无销量档排后),同档佣金高优先,id 兜底稳定分页
.order_by(nullslast(m.sale_volume_num.desc()), m.commission_percent.desc(), m.id)
.offset(start)
.limit(PAGE + 1)
).scalars().all()
page_order=lambda c: [
nullslast(c.sale_volume_num.desc()), c.commission_percent.desc(), c.id,
],
page=req.page, page_size=PAGE,
)
raws = _load_raws(db, ids)
except Exception: # noqa: BLE001
logger.exception("[feed] rec 库查询失败,降级返空")
return FeedResponse(items=[], has_next=False, page=req.page, status="degraded")
has_next = len(rows) > PAGE
cards: list[CouponCard] = []
for row in rows[:PAGE]:
try:
card = CouponCard.from_raw(row.raw or {})
except Exception: # noqa: BLE001
continue
if card.product_view_sign:
# 智能推荐不显示距离:库里的距离是相对城市默认点的(对用户无意义、且误导)。
# 置空后前端"距离 店名"那行只剩店名、自动顶到最左(店名移到原距离的位置)。
card.distance_text = None
card.distance_meters = None
cards.append(card)
cards = _cards_from_raws(raws, hide_distance=True)
if not cards and req.page == 1:
# 命中城市却 0 券:该城确无 ≥3% 券,或 ETL 灌的 city_id 与 city_dict 口径不一致。
logger.info("[feed] rec city_id=%s 命中 0 券(该城确无券?或 ETL/city_dict 的 city_id 口径不一致)", city_id)
@@ -282,51 +376,36 @@ def top_sales(req: TopSalesRequest, db: Session = Depends(get_db)) -> CouponList
if not city_id:
return CouponListResponse(items=[], has_next=False, search_id=None, status="degraded")
# 去重 + 排序 + 分页全在 SQL 做,每页只并解析当前页 ~20 条。
# (之前实现每翻一页都全表拉取 + 全量 from_raw 解析,翻页慢 → 客户端滑动卡顿/翻不动。)
# 去重 + 排序 + 分页全在 SQL 做,每页只回表并解析当前页 ~20 条(见 _paged_dedup_ids 的性能说明)
# 库为空(prod 刚部署 / ETL 未跑完)时返空 + status=empty,不崩;库查询异常降级 degraded。
conds = [
MeituanCoupon.sale_volume_num.isnot(None),
MeituanCoupon.city_id == city_id,
]
if req.platform is not None:
conds.append(MeituanCoupon.platform == req.platform)
try:
# 1) DISTINCT ON (dedup_key):每个去重键(品牌|名|价)只留销量最高那条(同销量再按佣金)
base = select(MeituanCoupon).where(
MeituanCoupon.sale_volume_num.isnot(None),
MeituanCoupon.city_id == city_id,
)
if req.platform is not None:
base = base.where(MeituanCoupon.platform == req.platform)
deduped = base.distinct(MeituanCoupon.dedup_key).order_by(
MeituanCoupon.dedup_key,
MeituanCoupon.sale_volume_num.desc(),
MeituanCoupon.commission_percent.desc(),
).subquery()
# 2) 对去重结果按销量降序分页;多取 1 条判断 has_next,只对本页做 from_raw
m = aliased(MeituanCoupon, deduped)
start = (req.page - 1) * req.page_size
rows = db.execute(
select(m)
ids, has_next = _paged_dedup_ids(
db,
conds=conds,
# 每个去重键(品牌|名|价)只留销量最高那条(同销量再按佣金)
dedup_order=[
MeituanCoupon.sale_volume_num.desc(),
MeituanCoupon.commission_percent.desc(),
],
# 加 id 作稳定 tiebreaker:同销量同佣金的并列项排序确定,避免跨页重复/漏项
.order_by(m.sale_volume_num.desc(), m.commission_percent.desc(), m.id)
.offset(start)
.limit(req.page_size + 1)
).scalars().all()
page_order=lambda c: [
c.sale_volume_num.desc(), c.commission_percent.desc(), c.id,
],
page=req.page, page_size=req.page_size,
)
raws = _load_raws(db, ids)
except Exception: # noqa: BLE001
logger.exception("[top-sales] 库查询失败,降级返空")
return CouponListResponse(items=[], has_next=False, search_id=None, status="degraded")
has_next = len(rows) > req.page_size
cards: list[CouponCard] = []
for row in rows[:req.page_size]:
try:
card = CouponCard.from_raw(row.raw or {})
except Exception: # noqa: BLE001
continue
if card.product_view_sign:
# 不显示距离:库里的距离是相对城市默认点的(对用户无意义、且误导)。
# 置空后前端"距离 店名"那行只剩店名、自动顶到最左(店名移到原距离的位置)。
# 逻辑与推荐流保持一致
card.distance_text = None
card.distance_meters = None
cards.append(card)
# 不显示距离:库里的距离是相对城市默认点的(对用户无意义、且误导),与推荐流口径一致。
cards = _cards_from_raws(raws, hide_distance=True)
if not cards and req.page == 1:
# 命中城市却 0 券:可能该城确无券,也可能 ETL 灌的 city_id 与 city_dict 口径不一致(静默降级的隐患)。
logger.info("[top-sales] city_id=%s 命中 0 券(该城确无券?或 ETL/city_dict 的 city_id 口径不一致)", city_id)
+8
View File
@@ -11,6 +11,7 @@ import logging
from fastapi import APIRouter, HTTPException, status
from app.api.deps import CurrentUser, DbSession
from app.repositories import risk as risk_repo
from app.repositories import task as crud_task
from app.schemas.welfare import TaskClaimResultOut, TaskListOut, TaskOut
@@ -36,6 +37,13 @@ def list_tasks(user: CurrentUser, db: DbSession) -> TaskListOut:
summary="领取任务奖励",
)
def claim_task(task_key: str, user: CurrentUser, db: DbSession) -> TaskClaimResultOut:
if risk_repo.is_restricted(
db,
subject_type="user",
subject_id=str(user.id),
scope=risk_repo.SCOPE_ECONOMIC_ACCOUNT,
):
raise HTTPException(status_code=403, detail="账号存在异常,该功能暂不可用")
try:
coin, balance = crud_task.claim_task(db, user.id, task_key)
except crud_task.UnknownTaskError as e:
+12 -8
View File
@@ -20,8 +20,9 @@ from app.core.rewards import (
WITHDRAW_MAX_CENTS,
WITHDRAW_MIN_CENTS,
)
from app.repositories import wallet as crud_wallet
from app.models.user import User
from app.repositories import risk as risk_repo
from app.repositories import wallet as crud_wallet
from app.schemas.welfare import (
BindWechatRequest,
BindWechatResultOut,
@@ -192,7 +193,7 @@ def withdraw_info(
wechat_bound=bool(u and u.wechat_openid),
wechat_nickname=u.wechat_nickname if u else None,
wechat_avatar_url=u.wechat_avatar_url if u else None,
transfer_auth_enabled=bool(auth and auth.state == "active"),
transfer_auth_enabled=bool(auth and auth.state == "active" and auth.authorization_id),
tiers=[WithdrawTierOut(**t) for t in crud_wallet.withdraw_tier_states(db, user.id, source)],
)
@@ -204,6 +205,13 @@ def withdraw_info(
dependencies=[Depends(rate_limit(5, 60, "withdraw"))], # IP 级粗限流;用户级未完成单限制在仓库层
)
def withdraw(req: WithdrawRequest, user: CurrentUser, db: DbSession) -> WithdrawResultOut:
if risk_repo.is_restricted(
db,
subject_type="user",
subject_id=str(user.id),
scope=risk_repo.SCOPE_ECONOMIC_ACCOUNT,
):
raise HTTPException(status_code=403, detail="账号存在异常,该功能暂不可用")
# 提现发起本身不调微信(打款在审核通过后),但仍要求微信支付已配置——否则审核通过也打不了款,提前拦
if not settings.wxpay_configured:
raise HTTPException(status_code=status.HTTP_503_SERVICE_UNAVAILABLE, detail="wechat pay not configured")
@@ -222,11 +230,6 @@ def withdraw(req: WithdrawRequest, user: CurrentUser, db: DbSession) -> Withdraw
) from e
except crud_wallet.WechatNotBoundError as e:
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="请先绑定微信") from e
except crud_wallet.WithdrawTooFrequentError as e:
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail="已有提现申请正在审核或打款中,请处理完成后再申请",
) from e
except crud_wallet.WithdrawTierUnavailableError as e:
# 福利页档位闸(7-9):次数满/已选其他额度。正常客户端已按 tiers 预拦,此处兜底防绕过。
raise HTTPException(status_code=status.HTTP_409_CONFLICT, detail="今日额度已达上限") from e
@@ -335,7 +338,8 @@ def open_transfer_auth(user: CurrentUser, db: DbSession) -> TransferAuthResultOu
def transfer_auth_status(user: CurrentUser, db: DbSession) -> TransferAuthStatusOut:
auth = crud_wallet.sync_transfer_auth(db, user.id)
state = auth.state if auth else "none"
return TransferAuthStatusOut(state=state, enabled=(state == "active"))
enabled = bool(auth and auth.state == "active" and auth.authorization_id)
return TransferAuthStatusOut(state=state, enabled=enabled)
@router.post(
+157
View File
@@ -0,0 +1,157 @@
"""客户端运行日志专用落盘 writer(独立于服务端 app-server.log)。
- 独占 logger "shagua.client_log" + 自己的 RotatingFileHandler,propagate=False 不污染 app-server.log
- 每条按白名单键(client_ts/level/trace_id/tag/msg)提顶层 + 其余并入 data封装,
json.dumps 成一行写出(钉死 SLS 索引列; spec §5)formatter %(message)s行本身
已是 JSON,不能再过 JsonFormatter 二次编码
- 滚动 20MB×10(env 可调),与服务日志同机制
依赖 --workers 1:RotatingFileHandler 多进程并发 doRollover 会损坏/丢日志; worker
前换 QueueHandler单写入者 / 外部 logrotate(copytruncate) / stdout journald
服务端补的字段(time/source/service/client_ip/device_id/...)事实,与客户端自述分开
`time` 用服务端接收时间作 SLS 主时间(客户端时钟不可信),client_ts 另存为可查字段
"""
from __future__ import annotations
import json
import logging
import os
from datetime import datetime
from logging.handlers import RotatingFileHandler
from pathlib import Path
# 仅这些客户端键提到输出行顶层;其余(含客户端自带 data)一律并入 data,防 SLS 索引列爆炸
_TOP_LEVEL_KEYS = ("client_ts", "level", "trace_id", "tag", "msg")
# trace_id/tag/level 是 SLS 索引字段(spec §8):给长度上限,防客户端塞超大值撑爆索引/抬升成本。
# (msg 另有字节截断;data 内的值不限,留待后续「服务端脱敏」knob。)
_MAX_LEVEL_LEN = 16
_MAX_TRACE_ID_LEN = 256
_MAX_TAG_LEN = 128
_logger: logging.Logger | None = None
def _max_msg_bytes() -> int:
# 每次调用现读 env(不设模块级常量):便于运行期调整 / 测试 monkeypatch,开销可忽略。
return int(os.getenv("APPLOG_MAX_MSG_BYTES", "8192"))
def _build_logger() -> logging.Logger:
lg = logging.getLogger("shagua.client_log") # 与仓库 shagua.* 业务 logger 命名一致
lg.setLevel(logging.INFO)
lg.propagate = False # 不冒泡到 root → 不写进 app-server.log
log_file = os.getenv("CLIENT_LOG_FILE") or str(
Path(os.getenv("LOG_DIR", "logs")) / "app-client.log"
)
Path(log_file).parent.mkdir(parents=True, exist_ok=True)
handler = RotatingFileHandler(
log_file,
maxBytes=int(os.getenv("CLIENT_LOG_MAX_BYTES", str(20 * 1024 * 1024))),
backupCount=int(os.getenv("CLIENT_LOG_BACKUP_COUNT", "10")),
encoding="utf-8",
)
handler.setFormatter(logging.Formatter("%(message)s")) # 行已是 JSON,不再包装
lg.handlers = [handler]
return lg
def get_logger() -> logging.Logger:
global _logger
if _logger is None:
_logger = _build_logger()
return _logger
def reset_client_logger() -> None:
"""测试用:关闭并丢弃当前 logger,使下次 get_logger 按当时 env 重建(切临时文件)。"""
global _logger
if _logger is not None:
for h in list(_logger.handlers):
h.close()
_logger.handlers = []
_logger = None
def _truncate_msg(msg: str) -> tuple[str, bool]:
raw = msg.encode("utf-8")
limit = _max_msg_bytes()
if len(raw) <= limit:
return msg, False
# 按字节截断后解码,忽略截断处半个多字节字符
return raw[:limit].decode("utf-8", "ignore") + "…[truncated]", True
def _build_line(
record: dict, *, meta: dict, client_ip: str, service: str, now_iso: str
) -> str:
out: dict = {
"time": now_iso,
"source": "client",
"service": service,
"client_ip": client_ip,
}
# 批级公共字段(非空才带)
for k in ("device_id", "user_id", "app_ver", "platform", "sent_at"):
v = meta.get(k)
if v is not None:
out[k] = v
# 白名单键提顶层(索引字段做长度上限 + 统一转 str,保证 SLS 里类型/大小可控)
if record.get("level") is not None:
out["level"] = str(record["level"])[:_MAX_LEVEL_LEN].upper()
if record.get("trace_id"):
out["trace_id"] = str(record["trace_id"])[:_MAX_TRACE_ID_LEN]
if record.get("tag"):
out["tag"] = str(record["tag"])[:_MAX_TAG_LEN]
if record.get("client_ts") is not None:
out["client_ts"] = record["client_ts"]
if record.get("msg") is not None:
msg, truncated = _truncate_msg(str(record["msg"]))
out["msg"] = msg
if truncated:
out["msg_truncated"] = True
# 其余键并入 data:先收白名单外的散键(兜底),再让客户端显式的 data 覆盖同名散键
# —— 显式 data 为准,不静默丢客户端明确给的值(保真)。
data: dict = {}
for k, v in record.items():
if k in _TOP_LEVEL_KEYS or k == "data":
continue
data[k] = v
client_data = record.get("data")
if isinstance(client_data, dict):
data.update(client_data)
if data:
out["data"] = data
return json.dumps(out, ensure_ascii=False, default=str)
def write_records(
records: list[dict], *, meta: dict, client_ip: str
) -> tuple[int, int]:
"""把一批客户端日志逐条写入专用文件。返回 (received, dropped)。
尽力而为(fire-and-forget):logger 初始化或单条写入失败只跳过并计 dropped,
不抛给上层端点因此永不因写日志而 500
"""
try:
lg = get_logger()
except Exception: # noqa: BLE001 — 初始化失败也不能让端点 500
logging.getLogger("shagua.applog").exception("client log writer init failed")
return 0, len(records)
service = os.getenv("CLIENT_LOG_SERVICE_NAME", "app-client")
# 一批共用同一「服务端接收时间」:降开销,且语义上是服务端「收到」而非逐条「处理」时间。
now_iso = datetime.now().strftime("%Y-%m-%dT%H:%M:%S.%f")[:-3]
received = dropped = 0
for rec in records:
try:
line = _build_line(
rec, meta=meta, client_ip=client_ip, service=service, now_iso=now_iso
)
lg.info(line)
received += 1
except Exception: # noqa: BLE001 — 坏条跳过,不影响其余
logging.getLogger("shagua.applog").exception(
"client log record dropped client_ip=%s", client_ip
)
dropped += 1
return received, dropped
+52
View File
@@ -141,6 +141,49 @@ class Settings(BaseSettings):
SMS_DAILY_LIMIT_PER_PHONE: int = 10 # 单手机号每日发送上限(防刷 + 控费)
SMS_MAX_VERIFY_ATTEMPTS: int = 5 # 单个验证码最多校验失败次数,超过即作废(防爆破)
# ===== 短信提供商(可切换:极光 / 阿里云号码认证 / 创蓝云智)=====
# jiguang(默认):本服务生成验证码,极光只负责下发,本地内存校验(自管码,现状不变)。
# aliyun:阿里云 dypns 号码认证,阿里云生成+下发+校验(Mode A);缺凭证时 /sms/* 返 503(优雅降级)。
# chuanglan:创蓝云智(253)模板短信,本服务生成码、创蓝只下发、本地校验(Mode B,与极光同);缺凭证 503。
SMS_PROVIDER: Literal["jiguang", "aliyun", "chuanglan"] = "jiguang"
ALIYUN_SMS_ACCESS_KEY_ID: str = ""
ALIYUN_SMS_ACCESS_KEY_SECRET: str = ""
ALIYUN_SMS_SIGN_NAME: str = "" # 系统赠送签名(自定义签名下发易失败)
ALIYUN_SMS_TEMPLATE_CODE: str = "" # 赠送模板 CODE(须与赠送签名搭配)
ALIYUN_SMS_SCHEME_NAME: str = "" # 方案名(可空=默认方案);send/check 共用避免不匹配
ALIYUN_SMS_ENDPOINT: str = "dypnsapi.aliyuncs.com"
ALIYUN_SMS_CODE_LENGTH: int = 6 # 验证码位数(CodeLength 4~8)
ALIYUN_SMS_VALID_TIME_SEC: int = 300 # 验证码有效期秒(ValidTime);短信内 min 文案 = //60
ALIYUN_SMS_INTERVAL_SEC: int = 60 # 单号发送频控秒(Interval);核验免费
ALIYUN_SMS_TIMEOUT_SEC: int = 15 # 阿里云 API 读/连超时秒
# --- 创蓝云智(253)模板短信,Mode B 自管码,httpx 直连 + HMAC 签名(见 docs/integrations/chuanglan/tpl-send.md)---
CHUANGLAN_SMS_ACCOUNT: str = "" # YZM 前缀验证码账号
CHUANGLAN_SMS_PASSWORD: str = "" # API 密码(仅用于本地算 HMAC 签名,不随请求上行)
CHUANGLAN_SMS_TEMPLATE_ID: str = "" # 模板 ID(控制台创建)
CHUANGLAN_SMS_SIGNATURE: str = "" # 短信签名文案【品牌】;模板已关联签名则留空
CHUANGLAN_SMS_ENDPOINT: str = "https://smssh.253.com/msg/sms/v2/tpl/send"
CHUANGLAN_SMS_TIMEOUT_SEC: int = 10 # httpx 读/连超时秒
@property
def aliyun_sms_configured(self) -> bool:
"""阿里云短信凭证齐全(缺则 SMS_PROVIDER=aliyun 时 /sms/* 返 503,而非启动崩)。"""
return bool(
self.ALIYUN_SMS_ACCESS_KEY_ID
and self.ALIYUN_SMS_ACCESS_KEY_SECRET
and self.ALIYUN_SMS_SIGN_NAME
and self.ALIYUN_SMS_TEMPLATE_CODE
)
@property
def chuanglan_sms_configured(self) -> bool:
"""创蓝短信凭证齐全(缺则 SMS_PROVIDER=chuanglan 时 /sms/send 返 503,而非启动崩)。"""
return bool(
self.CHUANGLAN_SMS_ACCOUNT
and self.CHUANGLAN_SMS_PASSWORD
and self.CHUANGLAN_SMS_TEMPLATE_ID
)
# ===== 测试账号(release 包全流程联调用)=====
# 配一个固定测试手机号,专供无 SIM 卡 / 不走一键登录时打通全流程:该号登录【免短信验证码】
# (real 模式下也跳过校验)、每次登录【强制重走新手引导】,并设【每日使用次数上限】防被人
@@ -377,6 +420,12 @@ class Settings(BaseSettings):
# 靠这个共享密钥头(X-Internal-Secret)校验,与 pricebot 侧 INTERNAL_API_SECRET 同值。
# 默认空 = 内部写端点关闭(返 503),启用前两边都要配上同一高熵串。
INTERNAL_API_SECRET: str = ""
# Current compare clients are persisted by server-side harvest. Repair any
# recent terminal rows left without LLM usage by transient upstream/auth failures.
LLM_COST_BACKFILL_ENABLED: bool = True
LLM_COST_BACKFILL_INTERVAL_SEC: int = 300
LLM_COST_BACKFILL_BATCH_SIZE: int = 100
LLM_COST_BACKFILL_LOOKBACK_DAYS: int = 30
# ===== 媒体文件(用户头像上传)=====
# 落盘根目录(data/ 已 gitignore,上传不进库);对外经 StaticFiles 挂在 MEDIA_URL_PREFIX。
@@ -384,6 +433,9 @@ class Settings(BaseSettings):
MEDIA_ROOT: str = "./data/media"
MEDIA_URL_PREFIX: str = "/media"
AVATAR_MAX_BYTES: int = 5 * 1024 * 1024 # 头像最大 5MB
# 运营后台上传的新手引导视频上限。视频比图片大一个量级,单独一档;
# ⚠️ 改大时同步放宽网关 client_max_body_size(实测 QA 4MiB / prod 32MiB),否则 nginx 先挡下。
GUIDE_VIDEO_MAX_BYTES: int = 100 * 1024 * 1024 # 引导视频最大 100MB
# ===== 邀请好友 =====
# 分享落地页(二维码 / 分享链接指向这里;扫码 → 落地页 → 引导浏览器下载 APK)。
+36
View File
@@ -11,6 +11,10 @@ from typing import Any
from app.core import rewards as r
RISK_SMS_HOURLY_THRESHOLD_KEY = "risk_sms_hourly_threshold"
RISK_ONECLICK_DAILY_THRESHOLD_KEY = "risk_oneclick_daily_threshold"
RISK_COMPARE_DAILY_THRESHOLD_KEY = "risk_compare_daily_threshold"
# type 约定(给前端渲染编辑控件用):int / int_list / dict_str_int / bool / enum
# hidden=True:仍是合法可配项(业务照常 get_value / admin 可经专用端点读写),但**不在通用
# 「系统配置」页渲染**(admin/routers/config.py:list_config 按此过滤)。用于把已下线/已改由
@@ -106,4 +110,36 @@ CONFIG_DEFS: dict[str, dict[str, Any]] = {
"default 兜底未登记的模型。改价只影响之后回填的新记录,历史记录用当时价格快照。"
),
},
# 风控监控页专用配置。复用 app_config 持久化,但不在通用「系统配置」页重复展示;
# 由 /admin/api/risk-monitor/rules 专用接口整组校验、审计和即时重算当前窗口。
RISK_SMS_HOURLY_THRESHOLD_KEY: {
"default": 5,
"label": "短信设备每小时告警阈值",
"group": "风控",
"type": "int",
"min": 1,
"max": 5,
"hidden": True,
"help": "同一设备在北京时间同一自然小时内成功下发短信达到该次数时告警;不得高于现有每小时 5 次的发送上限。",
},
RISK_ONECLICK_DAILY_THRESHOLD_KEY: {
"default": 20,
"label": "一键登录设备每日告警阈值",
"group": "风控",
"type": "int",
"min": 1,
"max": 100_000,
"hidden": True,
"help": "同一设备在北京时间同一自然日内一键登录成功和失败合计达到该次数时告警。",
},
RISK_COMPARE_DAILY_THRESHOLD_KEY: {
"default": 100,
"label": "比价账户每日告警阈值",
"group": "风控",
"type": "int",
"min": 1,
"max": 100,
"hidden": True,
"help": "同一账户在北京时间同一自然日内发起比价达到该次数时告警。",
},
}
+63
View File
@@ -0,0 +1,63 @@
"""Periodic repair worker for comparison records with missing LLM token cost."""
from __future__ import annotations
import asyncio
import contextlib
import logging
from app.core.config import settings
from app.services.comparison_llm_backfill import repair_missing_comparison_llm_costs
from app.services.pricebot_llm_calls import pricebot_llm_auth_ready
logger = logging.getLogger("shagua.llm_cost_backfill_worker")
async def _run_loop() -> None:
interval = max(60, int(settings.LLM_COST_BACKFILL_INTERVAL_SEC))
logger.info(
"LLM cost backfill worker started interval=%ss batch=%s lookback_days=%s",
interval,
settings.LLM_COST_BACKFILL_BATCH_SIZE,
settings.LLM_COST_BACKFILL_LOOKBACK_DAYS,
)
try:
while True:
try:
auth_ready = await asyncio.to_thread(pricebot_llm_auth_ready)
if auth_ready:
result = await asyncio.to_thread(
repair_missing_comparison_llm_costs,
limit=settings.LLM_COST_BACKFILL_BATCH_SIZE,
lookback_days=settings.LLM_COST_BACKFILL_LOOKBACK_DAYS,
)
logger.info("LLM cost backfill batch result=%s", result)
else:
logger.error(
"LLM cost backfill skipped: PriceBot internal auth is not ready"
)
except Exception: # noqa: BLE001
logger.exception("LLM cost backfill batch failed")
await asyncio.sleep(interval)
except asyncio.CancelledError:
logger.info("LLM cost backfill worker stopped")
raise
def start_llm_cost_backfill_worker() -> asyncio.Task | None:
if not settings.LLM_COST_BACKFILL_ENABLED:
logger.info("LLM cost backfill worker disabled")
return None
if not settings.INTERNAL_API_SECRET:
logger.warning(
"LLM cost backfill worker not started: INTERNAL_API_SECRET is empty"
)
return None
return asyncio.create_task(_run_loop(), name="llm-cost-backfill")
async def stop_llm_cost_backfill_worker(task: asyncio.Task | None) -> None:
if task is None:
return
task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await task
+51 -2
View File
@@ -22,13 +22,13 @@ from __future__ import annotations
import json
import logging
import os
import shutil
import sys
from contextvars import ContextVar
from datetime import datetime
from logging.handlers import RotatingFileHandler
from pathlib import Path
# 请求级 trace_id:入口(如 compare.py 透传壳)set 之后, 本请求上下文(含 run_in_threadpool
# 拷贝出去的线程)内所有日志自动带上。默认空串 = 非请求上下文(启动/后台 worker)。
trace_id_ctx: ContextVar[str] = ContextVar("trace_id", default="")
@@ -91,6 +91,55 @@ class TextFormatter(logging.Formatter):
return f"{base} trace={tid}" if tid else base
class SafeRotatingFileHandler(RotatingFileHandler):
"""Windows 下不会被外部句柄卡死的 RotatingFileHandler。
stdlib 轮转靠 rename 活动文件(app-server.log .1);Windows 只要有别的句柄(IDE 索引
app.admin.main 第二进程残留 --reload worker杀软扫描)开着它, rename WinError 32,
轮转永久卡死文件停在 maxBytes之后每条日志被丢这里 Windows 改用 copytruncate:把活动
文件拷进备份再通过自己的句柄原地清空, 从不 rename 活动文件, 故外部句柄开着也能转
POSIX(生产 Linux)rename 打开中的文件本就合法, 保留 stdlib 的原子轮转不变
代价:copytruncate 拷贝清空极窄窗口内并发写可能丢几行(仅跨进程;同进程 emit
handler 锁串行, 无此问题)对本地开发日志可接受
"""
def doRollover(self) -> None:
if os.name != "nt":
super().doRollover()
return
if self.stream is None:
self.stream = self._open()
else:
self.stream.flush()
try:
self._copytruncate_backups()
except OSError:
# 备份腾挪是尽力而为:任一备份被占用也绝不能挡住下面的清空, 否则活动文件继续涨、
# 轮转又卡死——那就白改了。
pass
# 通过自己独占的句柄原地清空:不涉及 rename, 外部只读句柄不受影响。
self.stream.seek(0)
self.stream.truncate()
self.stream.flush()
def _copytruncate_backups(self) -> None:
"""把 .N-1→.N 逐级腾挪, 再把活动文件拷到 .1(不动活动文件本身)。"""
if self.backupCount <= 0:
return
for i in range(self.backupCount - 1, 0, -1):
sfn = self.rotation_filename(f"{self.baseFilename}.{i}")
dfn = self.rotation_filename(f"{self.baseFilename}.{i + 1}")
if os.path.exists(sfn):
if os.path.exists(dfn):
os.remove(dfn)
os.replace(sfn, dfn)
dfn = self.rotation_filename(f"{self.baseFilename}.1")
if os.path.exists(dfn):
os.remove(dfn)
shutil.copyfile(self.baseFilename, dfn)
_CONFIGURED = False
@@ -126,7 +175,7 @@ def setup_logging(debug: bool = False) -> None:
Path(os.getenv("LOG_DIR", "logs")) / "app-server.log"
)
Path(log_file).parent.mkdir(parents=True, exist_ok=True)
file_handler = RotatingFileHandler(
file_handler = SafeRotatingFileHandler(
log_file, maxBytes=10 * 1024 * 1024, backupCount=5, encoding="utf-8",
)
file_handler.setFormatter(JsonFormatter(service))
+34
View File
@@ -77,6 +77,35 @@ def save_feedback_qr(data: bytes) -> str:
return _save_named("feedback_qr", "qr", data)
def _sniff_video_ext(data: bytes) -> str | None:
"""按魔数判定视频类型,返回扩展名;非支持类型返回 None。
只认 MP4 家族(ISO BMFF):`....ftyp` 在偏移 4Android ExoPlayer 与浏览器 <video>
都稳吃 H.264/AAC mp4;放开 mkv/avi 只会让端上放不出来,不如在入口就挡掉
"""
if len(data) >= 12 and data[4:8] == b"ftyp":
return ".mp4"
return None
def save_guide_video(data: bytes) -> str:
"""保存新手引导视频(运营后台上传的运营素材),返回相对 URL(`/media/guide_video/<file>`)。
与图片分开一套校验:体积上限走 [settings.GUIDE_VIDEO_MAX_BYTES],类型只认 MP4
"""
if not data:
raise MediaError("空文件")
limit = settings.GUIDE_VIDEO_MAX_BYTES
if len(data) > limit:
raise MediaError(f"视频过大(上限 {limit // (1024 * 1024)}MB)")
if _sniff_video_ext(data) is None:
raise MediaError("仅支持 MP4 视频(H.264 编码)")
fname = f"guide_{secrets.token_hex(8)}.mp4"
(_media_dir("guide_video") / fname).write_bytes(data)
return f"{settings.MEDIA_URL_PREFIX}/guide_video/{fname}"
def save_cps_image(admin_id: int, data: bytes) -> str:
"""保存 CPS 活动落地页图,返回相对 URL(`/media/cps/<file>`)。admin_id 入文件名便于追溯。"""
return _save_image("cps", admin_id, data)
@@ -116,3 +145,8 @@ def delete_avatar(url: str | None) -> None:
def delete_feedback_qr(url: str | None) -> None:
"""删除本服务托管的旧反馈页二维码文件;外部 URL 或空值不处理。"""
_delete_managed("feedback_qr", url)
def delete_guide_video(url: str | None) -> None:
"""删除本服务托管的旧新手引导视频文件;外部 URL 或空值不处理。"""
_delete_managed("guide_video", url)
+40 -5
View File
@@ -11,6 +11,7 @@ import hashlib
import hmac
import json
import logging
import threading
import time
from typing import Any
@@ -25,6 +26,44 @@ class MeituanCpsError(Exception):
"""美团 CPS 接口调用失败。"""
# ────────────────────── 共享 httpx.Client(连接池 + keep-alive) ──────────────────────
# 原实现每次 _call 都 `with httpx.Client(...)` 新建再关掉,等于每发一次请求就:
# ① 重建一套 SSL 上下文(加载 certifi CA,实测几百 ms) ② 重做一次 TLS 握手 ③ 用完即弃连接。
# 首页 feed 一次翻页要向美团发好几次请求(外卖/到店两路,「距离最近」还要续页),这份固定开销被成倍放大,
# 是「滑到底部加载很慢」的一大块。改成进程内单例:握手一次、后续走 keep-alive 复用。
# httpx.Client 本身线程安全,可被 feed 的 ThreadPoolExecutor 两条抓取线程共用。
_client: httpx.Client | None = None
_client_lock = threading.Lock()
def get_client() -> httpx.Client:
"""取美团 CPS 共享 client(幂等,懒建)。lifespan 启动时预热,把建 SSL 的成本摊到启动。"""
global _client
if _client is None:
with _client_lock:
if _client is None:
# 走 MT_CPS_PROXY(本机开发直连会 SSL EOF,必须走代理;线上留空=直连)。
# trust_env=False:不读进程环境的 HTTP_PROXY,只认配置,避免被错误/失效代理误导。
_client = httpx.Client(
proxy=settings.MT_CPS_PROXY or None,
trust_env=False,
timeout=settings.MT_CPS_TIMEOUT_SEC,
# 池子留足:feed 每个请求会起 2 条抓取线程,并发用户多时别在池上排队
# (排满会等到 MT_CPS_TIMEOUT_SEC 抛 PoolTimeout,表现成"又慢又降级")。
limits=httpx.Limits(max_keepalive_connections=16, max_connections=32),
)
return _client
def close_client() -> None:
"""lifespan 关停时调,优雅关连接池。"""
global _client
with _client_lock:
if _client is not None:
_client.close()
_client = None
def _content_md5(body: bytes) -> str:
return base64.b64encode(hashlib.md5(body).digest()).decode()
@@ -62,12 +101,8 @@ def _call(path: str, body_obj: dict[str, Any]) -> dict[str, Any]:
}
url = f"{settings.MT_CPS_HOST}{path}"
# 美团调用走 MT_CPS_PROXY(本机开发直连会 SSL EOF,必须走代理;线上留空=直连)。
# trust_env=False:不读进程环境的 HTTP_PROXY,只认配置,避免被错误/失效代理误导。
proxy = settings.MT_CPS_PROXY or None
try:
with httpx.Client(proxy=proxy, trust_env=False, timeout=settings.MT_CPS_TIMEOUT_SEC) as client:
resp = client.post(url, content=body, headers=headers)
resp = get_client().post(url, content=body, headers=headers)
except httpx.HTTPError as e:
logger.exception("[MT] http error calling %s", url)
raise MeituanCpsError(f"meituan http error: {e}") from e
+37
View File
@@ -0,0 +1,37 @@
"""短信验证码服务 —— provider 分派入口。
对外只暴露 `send_code` / `verify_code` / `SmsError`,api 层无需关心用哪个 provider
provider `settings.SMS_PROVIDER` 选择(**每次调用读取**,支持运行时切换 + 灰度回退):
- `jiguang`(默认):自管码(本服务生成内存存/校验,极光只发) [jiguang.py](jiguang.py)
- `aliyun`:阿里云号码认证(阿里云生成+下发+校验,Mode A) [aliyun.py](aliyun.py)
- `chuanglan`:创蓝云智(253)模板短信,自管码 Mode B(本服务生成内存存/校验,创蓝只发) [chuanglan.py](chuanglan.py)
mock(`SMS_MOCK=true`)与各 provider 的行为差异都封在 provider 内部;本层只做路由
拆包前本模块是单文件 `sms.py`;拆包后极光逻辑迁入 `jiguang` 子模块,行为零改动
"""
from __future__ import annotations
from app.core.config import settings
from . import aliyun, chuanglan, jiguang
from .base import SmsError
__all__ = ["SmsError", "send_code", "verify_code"]
# provider 名 -> 模块;未知/缺省值回退 jiguang(默认兜底,防误配把登录打挂)。
_PROVIDERS = {"aliyun": aliyun, "chuanglan": chuanglan}
def _provider():
"""按配置选 provider 模块(每次调用读 settings,支持运行时切换 / 测试注入)。"""
return _PROVIDERS.get(settings.SMS_PROVIDER, jiguang)
def send_code(phone: str) -> int:
"""发送验证码,返回距下次可发的冷却秒数;失败抛 SmsError。委托给当前 provider。"""
return _provider().send_code(phone)
def verify_code(phone: str, code: str) -> bool:
"""校验验证码,返回是否通过;provider 异常降级抛 SmsError。委托给当前 provider。"""
return _provider().verify_code(phone, code)
+193
View File
@@ -0,0 +1,193 @@
"""阿里云号码认证(dypns)短信 provider —— Mode A(阿里云托管验证码)。
与极光(自管码)最大不同:**本服务不生成/不存储验证码**,验证码由阿里云生成+存储+下发+校验
- 发码: SendSmsVerifyCode,TemplateParam `{"code":"##code##","min":...}` 占位,阿里云生成
- 校验: CheckSmsVerifyCode,阿里云返回 PASS / UNKNOWN核验免费
天然消除极光路径内存存码 worker 不共享的技术债(发码/校验可落不同 worker,阿里云统一裁决)
**唯一本地态**:per-phone 连续失败计数(`_verify_attempts`),用于复刻极光单码失败
`SMS_MAX_VERIFY_ATTEMPTS` 次即作废的防爆破语义 刻意与极光一致,避免两 provider 行为不同
导致排查困惑其多 worker 降级特性与极光现状同级;另有 API 层登录频控(设备+IP)做硬兜底
单号发送频控(冷却)交给阿里云 `Interval` 参数(命中FREQUENCY_FAIL429),本地不再维护冷却
SDK 交互隔离在 `_call_send` / `_call_check` 两个薄封装(惰性 import + 惰性建 client,仿 wxpay
惰性加载),单测 monkeypatch 这两个即可,不触真 SDK / 网络
"""
from __future__ import annotations
import json
import logging
import time
from threading import Lock
from app.core.config import settings
from .base import SmsError, mock_verify
logger = logging.getLogger("shagua.sms.aliyun")
# 阿里云路径唯一本地态:per-phone 连续失败次数(与极光同语义,防爆破)。
_verify_attempts: dict[str, int] = {} # phone -> 连续失败次数
_verify_seen: dict[str, float] = {} # phone -> 最近触碰 epoch(仅供 GC 老化)
_lock = Lock()
_GC_THRESHOLD = 10000 # 超此阈值,send 时顺手清老于验证码有效期的计数(仿极光 _gc)
# 发码错误码 → (HTTP 码, 用户提示)。未列出的一律 503(供应商不可用)。
_SEND_ERRORS: dict[str, tuple[int, str]] = {
"MOBILE_NUMBER_ILLEGAL": (400, "手机号无效"),
"BUSINESS_LIMIT_CONTROL": (429, "今日发送次数过多,请明天再试"),
"FREQUENCY_FAIL": (429, "发送过于频繁,请稍后再试"),
}
# 需运维介入的配置/开通类错误:打 critical 日志(融合认证未开通 / 参数非法)。
_SEND_CRITICAL_CODES = frozenset({"FUNCTION_NOT_OPENED", "INVALID_PARAMETERS"})
_client = None # 惰性构建的 SDK client(模块级缓存)
# ============================ 对外:发码 / 校验 ============================
def send_code(phone: str) -> int:
"""发送验证码(阿里云生成+下发)。
Returns: 距下次可发的秒数(= ALIYUN_SMS_INTERVAL_SEC,冷却由阿里云 Interval 侧执行)
Raises: SmsError(手机号无效 400 / 过频·天级流控 429 / 未配置·未开通·其他 503)
"""
if settings.SMS_MOCK:
logger.info("[SMS-aliyun-MOCK] to %s**** (不真发)", phone[:3])
return settings.ALIYUN_SMS_INTERVAL_SEC
if not settings.aliyun_sms_configured:
raise SmsError("短信服务未配置(缺阿里云凭证)", status_code=503)
result = _call_send(phone) # 传输/SDK 异常在内部抛 SmsError(503)
if result["success"] and result["code"] == "OK":
now = time.time()
with _lock:
_gc(now) # 顺手清老计数(超阈值才扫)
_verify_attempts.pop(phone, None) # 新码 = 新失败预算
_verify_seen.pop(phone, None)
logger.info("[SMS-aliyun] sent to %s****", phone[:3])
return settings.ALIYUN_SMS_INTERVAL_SEC
code = result["code"]
logger.error("[SMS-aliyun] send failed code=%s msg=%s", code, result["message"])
if code in _SEND_CRITICAL_CODES:
logger.critical("[SMS-aliyun] %s —— 需运维处理(融合认证未开通 / 参数非法)", code)
status, msg = _SEND_ERRORS.get(code, (503, "短信服务暂不可用,请稍后重试"))
raise SmsError(msg, status_code=status)
def verify_code(phone: str, code: str) -> bool:
"""校验验证码(阿里云裁决)。
- **mock**:放行任意 N 位数字(provider 无关,同极光)
- **real**:先查本地失败计数(达上限即本地作废,不调阿里云,与极光一致) CheckSmsVerifyCode:
PASS 清计数返 True(一次性);UNKNOWN 计数 +1 False;接口异常抛 SmsError(503)
"""
if settings.SMS_MOCK:
ok = mock_verify(code)
logger.info("[SMS-aliyun-MOCK] verify %s for %s****", "ok" if ok else "fail", phone[:3])
return ok
# 失败计数是 best-effort:网络调用不持锁(不能锁跨 IO),故并发下同号可能多放行个位数次。
# 无碍——API 层登录频控(设备+IP 5/时)是硬上限,阿里云码有效期 + DuplicatePolicy 亦兜底。
with _lock:
if _verify_attempts.get(phone, 0) >= settings.SMS_MAX_VERIFY_ATTEMPTS:
return False # 已作废:保持计数(直到 send_code 重置),与极光「达上限即作废」一致
result = _call_check(phone, code) # 传输/SDK 异常在内部抛 SmsError(503)
if not (result["success"] and result["code"] == "OK"):
# 接口层失败(非码错):降级 503,别误报「验证码错误」(400),便于区分排查。
logger.error("[SMS-aliyun] check failed code=%s msg=%s", result["code"], result["message"])
raise SmsError("短信服务暂不可用,请稍后重试", status_code=503)
if result["verify_result"] == "PASS":
with _lock:
_verify_attempts.pop(phone, None) # 验过即清(一次性)
_verify_seen.pop(phone, None)
return True
# UNKNOWN:码错 / 过期 → 失败计数 +1(累计到上限即作废)
with _lock:
_verify_attempts[phone] = _verify_attempts.get(phone, 0) + 1
_verify_seen[phone] = time.time()
return False
def _gc(now: float) -> None:
"""超阈值时清理老于验证码有效期的失败计数(码早已在阿里云侧失效,计数无意义)。仅持锁调用。"""
if len(_verify_attempts) <= _GC_THRESHOLD:
return
cutoff = now - settings.ALIYUN_SMS_VALID_TIME_SEC
for p in [p for p, ts in _verify_seen.items() if ts < cutoff]:
_verify_attempts.pop(p, None)
_verify_seen.pop(p, None)
# ============================ SDK 接缝(单测 monkeypatch 这两个)============================
def _get_client():
"""惰性构建 dypns SDK client(仿 wxpay 惰性加载:jiguang-only 部署不加载 alibabacloud)。"""
global _client
if _client is None:
from alibabacloud_dypnsapi20170525.client import Client
from alibabacloud_tea_openapi import models as open_api_models
cfg = open_api_models.Config(
access_key_id=settings.ALIYUN_SMS_ACCESS_KEY_ID,
access_key_secret=settings.ALIYUN_SMS_ACCESS_KEY_SECRET,
read_timeout=settings.ALIYUN_SMS_TIMEOUT_SEC * 1000, # SDK 单位 ms
connect_timeout=settings.ALIYUN_SMS_TIMEOUT_SEC * 1000,
)
cfg.endpoint = settings.ALIYUN_SMS_ENDPOINT
_client = Client(cfg)
return _client
def _call_send(phone: str) -> dict:
"""调 SendSmsVerifyCode。返回归一化 {success, code, message};import/建 client/调用 任一失败抛 SmsError(503)。"""
valid_min = max(1, settings.ALIYUN_SMS_VALID_TIME_SEC // 60)
template_param = json.dumps({"code": "##code##", "min": str(valid_min)}, ensure_ascii=False)
try:
# import + 建 req + 调用 全在 try 内:任一 provider 侧失败都归一成 503(保「provider 出问题→503」不变式)
from alibabacloud_dypnsapi20170525 import models as dypns_models
req = dypns_models.SendSmsVerifyCodeRequest(
phone_number=phone,
sign_name=settings.ALIYUN_SMS_SIGN_NAME,
template_code=settings.ALIYUN_SMS_TEMPLATE_CODE,
template_param=template_param,
code_length=settings.ALIYUN_SMS_CODE_LENGTH,
valid_time=settings.ALIYUN_SMS_VALID_TIME_SEC,
interval=settings.ALIYUN_SMS_INTERVAL_SEC,
scheme_name=settings.ALIYUN_SMS_SCHEME_NAME or None,
)
body = _get_client().send_sms_verify_code(req).body
except Exception as e:
logger.exception("[SMS-aliyun] send_sms_verify_code 调用异常 phone=%s****", phone[:3])
raise SmsError("短信服务暂不可用,请稍后重试", status_code=503) from e
return {"success": bool(body.success), "code": body.code, "message": body.message}
def _call_check(phone: str, code: str) -> dict:
"""调 CheckSmsVerifyCode。返回归一化 {success, code, message, verify_result};import/建 client/调用 任一失败抛 SmsError(503)。"""
try:
# import + 建 req + 调用 全在 try 内:任一 provider 侧失败都归一成 503(保「provider 出问题→503」不变式)
from alibabacloud_dypnsapi20170525 import models as dypns_models
req = dypns_models.CheckSmsVerifyCodeRequest(
phone_number=phone,
verify_code=code,
scheme_name=settings.ALIYUN_SMS_SCHEME_NAME or None,
)
body = _get_client().check_sms_verify_code(req).body
except Exception as e:
logger.exception("[SMS-aliyun] check_sms_verify_code 调用异常 phone=%s****", phone[:3])
raise SmsError("短信服务暂不可用,请稍后重试", status_code=503) from e
verify_result = getattr(body.model, "verify_result", None) if body.model else None
return {
"success": bool(body.success),
"code": body.code,
"message": body.message,
"verify_result": verify_result,
}
+23
View File
@@ -0,0 +1,23 @@
"""短信 provider 共享基座:业务异常 + provider 无关的 mock 校验。
provider(jiguang / aliyun) `from .base import SmsError`,api 层也从包入口拿到同一个
`SmsError` 保证无论用哪个 provider,异常类型与 HTTP 码映射语义一致
"""
from __future__ import annotations
from app.core.config import settings
class SmsError(Exception):
"""业务异常。`status_code` 决定 api 层翻成哪个 HTTP 码:
过频/每日超限 = 429(客户端等会再来),供应商不可用 = 503,手机号无效 = 400
"""
def __init__(self, message: str, status_code: int = 429) -> None:
super().__init__(message)
self.status_code = status_code
def mock_verify(code: str) -> bool:
"""mock 模式校验:放行任意 SMS_CODE_LENGTH 位数字(provider 无关,测试/开发便利,不真校验)。"""
return len(code) == settings.SMS_CODE_LENGTH and code.isdigit()
+224
View File
@@ -0,0 +1,224 @@
"""创蓝云智(253)短信 provider(自管码 Mode B)。
创蓝 `tpl/send` v2 **纯发送网关**(本服务生成码 放入 templateParamJson 创蓝只下发,
无校验接口),故与极光同为 **Mode B**:本服务生成/存储/校验验证码,创蓝只负责发
**本模块的存码/冷却/一次性/防爆破/GC 机器与 [jiguang.py](jiguang.py) 是刻意的隔离复制**
(设计见 docs/superpowers/specs/2026-07-26-chuanglan-sms-verify-design.md):极光文件一行不动
零回归风险于登录关键路径的默认 provider;代价是两处 Mode B 并发逻辑重复,改动需同步唯一新逻辑
`_send_via_chuanglan`(HMAC-SHA256 签名 + httpx POST + 错误码映射)
两种运行模式由 `SMS_MOCK` 切换:
- **mock**(开发/测试,默认):不真发,验证码打日志;校验放行任意 N 位数字
- **real**(`SMS_MOCK=false` `SMS_PROVIDER=chuanglan`):`secrets` 生成码 调创蓝 `tpl/send`
下发(HMAC 签名,password 仅本地算签不上行) 校验比对本地存码(一次性 / 过期 / 防爆破)
验证码存储:**进程内存**( worker 够用, worker 不共享,与极光同级技术债)防刷同极光:
单号 `SMS_SEND_INTERVAL_SEC` 冷却(本文件)+ 单设备/IP 频控(api )+ 单码失败 `SMS_MAX_VERIFY_ATTEMPTS`
次即作废运维侧另需在创蓝控制台配 **IP 白名单**(否则 117)接口调研见 docs/integrations/chuanglan/tpl-send.md
"""
from __future__ import annotations
import hashlib
import hmac
import json
import logging
import secrets
import time
from dataclasses import dataclass
from threading import Lock
import httpx
from app.core.config import settings
from .base import SmsError, mock_verify
logger = logging.getLogger("shagua.sms.chuanglan")
@dataclass
class _CodeRecord:
code: str
expires_at: float
attempts: int = 0
# 进程内存(单 worker 有效;多 worker 不共享,见模块 docstring)。与极光同结构。
_codes: dict[str, _CodeRecord] = {} # phone -> 当前有效验证码
_last_sent: dict[str, float] = {} # phone -> 上次发送 epoch(冷却)
_lock = Lock()
_GC_THRESHOLD = 10000 # 任一内存 dict 超此阈值,send 时顺手清过期项(防无限增长)
# 发码错误码(创蓝 `code`)→ (HTTP 码, 用户提示)。未列出的一律 503(供应商不可用)。
_SEND_ERRORS: dict[str, tuple[int, str]] = {
"103": (429, "发送过于频繁,请稍后再试"), # 提交速度过快
"107": (400, "手机号无效"), # 手机号码错误
}
# 需运维介入的配置/开通/余额类错误:打 critical 日志(仍归 503)。
_SEND_CRITICAL_CODES = frozenset({
"109", # 无发送量/余额不足
"117", # IP 未加白名单
"102", # 密码错误
"116", # 签名不合法
"124", # 模板内容不匹配
"152", # 模板不存在
"101", # 账号不存在
"118", # 无发送权限
})
def _gen_code() -> str:
"""生成 N 位数字验证码(用 secrets 而非 random;允许前导 0)。"""
return "".join(secrets.choice("0123456789") for _ in range(settings.SMS_CODE_LENGTH))
def _gc(now: float) -> None:
"""顺手清理过期内存项,防两个 dict 无限增长。仅在持锁时调用,且某 dict 超阈值才扫它。"""
if len(_codes) > _GC_THRESHOLD:
for p in [p for p, r in _codes.items() if now > r.expires_at]:
_codes.pop(p, None)
if len(_last_sent) > _GC_THRESHOLD:
cutoff = now - settings.SMS_SEND_INTERVAL_SEC
for p in [p for p, ts in _last_sent.items() if ts < cutoff]:
_last_sent.pop(p, None)
def send_code(phone: str) -> int:
"""发送验证码。
Returns: 距下次可发的秒数(= SMS_SEND_INTERVAL_SEC)
Raises: SmsError(过频 429 / 手机号无效 400 / 供应商失败 503)
"""
now = time.time()
# --- lock 内:防刷检查 + 预占(防并发重复发烧钱)---
with _lock:
_gc(now) # 顺手清过期内存(超阈值才扫)
elapsed = now - _last_sent.get(phone, 0.0)
if elapsed < settings.SMS_SEND_INTERVAL_SEC:
remain = int(settings.SMS_SEND_INTERVAL_SEC - elapsed)
raise SmsError(f"发送过于频繁,请 {remain}s 后再试")
code = _gen_code()
# 预占:先记冷却/存码,释放锁后再发网络(发失败保留冷却,见下)
_last_sent[phone] = now
_codes[phone] = _CodeRecord(code=code, expires_at=now + settings.SMS_CODE_TTL_SEC)
# --- lock 外:真正发送(网络 IO 不持锁)---
try:
if settings.SMS_MOCK:
logger.info("[SMS-chuanglan-MOCK] to %s**** code=%s (不真发)", phone[:3], code)
else:
_send_via_chuanglan(phone, code)
logger.info("[SMS-chuanglan] sent to %s****", phone[:3])
except Exception as e:
# 发送失败:**保留冷却**(失败也限速,挡住余额不足/签名失效时前端重试狂打),
# 只清掉没发出去的码(用户收不到,留着无意义且占内存)。
with _lock:
_codes.pop(phone, None)
if isinstance(e, SmsError):
raise
logger.exception("[SMS-chuanglan] send failed phone=%s****", phone[:3])
raise SmsError("验证码发送失败,请稍后重试", status_code=503) from e
return settings.SMS_SEND_INTERVAL_SEC
def verify_code(phone: str, code: str) -> bool:
"""校验验证码。
- **mock 模式**:放行任意 N 位数字(测试/开发便利,不真校验)
- **real 模式**:比对本服务存的码,匹配即作废(一次性);失败累计到上限也作废(防爆破)
"""
if settings.SMS_MOCK:
ok = mock_verify(code)
logger.info("[SMS-chuanglan-MOCK] verify %s for %s****", "ok" if ok else "fail", phone[:3])
return ok
with _lock:
rec = _codes.get(phone)
if rec is None:
return False
if time.time() > rec.expires_at:
_codes.pop(phone, None)
return False
if rec.attempts >= settings.SMS_MAX_VERIFY_ATTEMPTS:
_codes.pop(phone, None) # 试错过多,作废
return False
if secrets.compare_digest(code.encode("utf-8"), rec.code.encode("utf-8")):
_codes.pop(phone, None) # 验过即作废
return True
rec.attempts += 1
return False
# ============================ 发送接缝(单测 monkeypatch 这两个 / httpx.post)============================
def _sign(password: str, timestamp: str, nonce: str) -> str:
"""创蓝 HMAC-SHA256 签名:key=md5(password),msg=sorted([md5pwd,ts,nonce]) 拼接去空白,输出小写 hex。"""
md5pwd = hashlib.md5(password.encode()).hexdigest() # 32 位小写 hex
raw = "".join(sorted([md5pwd, timestamp, nonce])) # 字典序升序,无分隔符拼接
raw = "".join(raw.split()) # 去所有空白(faithful;三段本无空白)
return hmac.new(md5pwd.encode(), raw.encode(), hashlib.sha256).hexdigest()
def _call_chuanglan(phone: str, code: str) -> dict:
"""组装 + 签名 + POST 创蓝 tpl/send,返回解析后的响应 dict。
传输错误 / HTTP200 / 响应非 JSON 一律抛 SmsError(503)(provider 出问题503不变式);
业务码( 000000)由调用方 `_send_via_chuanglan` 判读password 只用于算签,不入 body
"""
timestamp = str(int(time.time()))
nonce = secrets.token_hex(16) # 32 位 hex
body = {
"account": settings.CHUANGLAN_SMS_ACCOUNT,
"timestamp": timestamp,
"nonce": nonce,
"phoneNumbers": phone,
"templateId": settings.CHUANGLAN_SMS_TEMPLATE_ID,
"templateParamJson": json.dumps([{"param1": code}]),
}
if settings.CHUANGLAN_SMS_SIGNATURE:
body["signature"] = settings.CHUANGLAN_SMS_SIGNATURE
headers = {
"Content-Type": "application/json",
"X-QA-Hmac-Signature": _sign(settings.CHUANGLAN_SMS_PASSWORD, timestamp, nonce),
}
try:
resp = httpx.post(
settings.CHUANGLAN_SMS_ENDPOINT,
json=body,
headers=headers,
timeout=settings.CHUANGLAN_SMS_TIMEOUT_SEC,
)
except httpx.HTTPError as e:
logger.exception("[SMS-chuanglan] 网络错误 phone=%s****", phone[:3])
raise SmsError("短信服务暂不可用,请稍后重试", status_code=503) from e
if resp.status_code != 200:
logger.error("[SMS-chuanglan] http=%s body=%s", resp.status_code, resp.text[:200])
raise SmsError("短信服务暂不可用,请稍后重试", status_code=503)
try:
return resp.json()
except Exception as e:
logger.error("[SMS-chuanglan] 响应非 JSON: %s", resp.text[:200])
raise SmsError("短信服务暂不可用,请稍后重试", status_code=503) from e
def _send_via_chuanglan(phone: str, code: str) -> None:
"""调创蓝 tpl/send 发送。成功静默返回;失败按错误码映射抛 SmsError。"""
if not settings.chuanglan_sms_configured:
raise SmsError("短信服务未配置(缺创蓝 account/password/templateId)", status_code=503)
result = _call_chuanglan(phone, code) # 传输/非200/解析异常在内部抛 SmsError(503)
rcode = str(result.get("code"))
if rcode == "000000":
return
emsg = result.get("errorMsg") or ""
logger.error("[SMS-chuanglan] send failed code=%s msg=%s", rcode, emsg)
if rcode in _SEND_CRITICAL_CODES:
logger.critical("[SMS-chuanglan] %s —— 需运维处理(余额/IP白名单/密码/签名/模板/账号)", rcode)
status, msg = _SEND_ERRORS.get(rcode, (503, "短信服务暂不可用,请稍后重试"))
raise SmsError(msg, status_code=status)
@@ -1,15 +1,15 @@
"""短信验证码服务
"""极光短信 provider(自管码 Mode B)
两种运行模式由 `SMS_MOCK` 切换:
- **mock**(开发/测试,默认):不真发短信,验证码打到日志;校验**放行任意 N 位数字**
(测试/开发便利)真实校验逻辑(比对存码 / 一次性 / 防爆破) real 分支 + 单测覆盖
- **real**(生产 `SMS_MOCK=false`):本服务生成 N 位验证码 调极光短信 REST
`/v1/messages` 发送(自定义验证码模式,极光只负责发,code 由本服务生成/保管/
- **real**(生产 `SMS_MOCK=false` `SMS_PROVIDER=jiguang`):本服务生成 N 位验证码 调极光
短信 REST `/v1/messages` 发送(自定义验证码模式,极光只负责发,code 由本服务生成/保管/
校验) 鉴权复用极光一键登录的 `JG_APP_KEY`/`JG_MASTER_SECRET`(同一极光应用)
验证码存储:**进程内存**( worker uvicorn 够用)重启丢失(用户重发即可)
worker / 多机时内存不共享 冷却校验都会失效,届时迁移到 DB/Redis
docs/待办与技术债.md
worker / 多机时内存不共享 冷却校验都会失效,届时迁移到 DB/Redis(或改用 aliyun provider,
其验证码由阿里云托管无本地存码) docs/待办与技术债.md
防刷两层(短信花钱 + `/sms/send` 在登录前无法 JWT 鉴权):
1. 单号 `SMS_SEND_INTERVAL_SEC` 冷却(本文件)
@@ -34,17 +34,9 @@ import httpx
from app.core.config import settings
logger = logging.getLogger("shagua.sms")
from .base import SmsError, mock_verify
class SmsError(Exception):
"""业务异常。`status_code` 决定 api 层翻成哪个 HTTP 码:
过频/每日超限 = 429(客户端等会再来),供应商不可用 = 503,手机号无效 = 400
"""
def __init__(self, message: str, status_code: int = 429) -> None:
super().__init__(message)
self.status_code = status_code
logger = logging.getLogger("shagua.sms.jiguang")
@dataclass
@@ -126,7 +118,7 @@ def verify_code(phone: str, code: str) -> bool:
- **real 模式**:比对本服务存的码,匹配即作废(一次性);失败累计到上限也作废(防爆破)
"""
if settings.SMS_MOCK:
ok = len(code) == settings.SMS_CODE_LENGTH and code.isdigit()
ok = mock_verify(code)
logger.info("[SMS-MOCK] verify %s for %s****", "ok" if ok else "fail", phone[:3])
return ok
+16
View File
@@ -21,6 +21,7 @@ from app.api.internal.price import router as internal_price_router
from app.api.internal.store import router as internal_store_router
from app.api.v1.ad import router as ad_router
from app.api.v1.analytics import router as analytics_router
from app.api.v1.applog import router as applog_router
from app.api.v1.auth import router as auth_router
from app.api.v1.compare import router as compare_router
from app.api.v1.compare_milestone import router as compare_milestone_router
@@ -29,6 +30,7 @@ from app.api.v1.coupon import router as coupon_router
from app.api.v1.cps_redirect import router as cps_redirect_router
from app.api.v1.device import router as device_router
from app.api.v1.feedback import router as feedback_router
from app.api.v1.guide_video import router as guide_video_router
from app.api.v1.invite import router as invite_router
from app.api.v1.meituan import router as meituan_router
from app.api.v1.notifications import router as notifications_router
@@ -59,6 +61,10 @@ from app.core.inactivity_reset_worker import (
start_inactivity_reset_worker,
stop_inactivity_reset_worker,
)
from app.core.llm_cost_backfill_worker import (
start_llm_cost_backfill_worker,
stop_llm_cost_backfill_worker,
)
from app.core.logging import setup_logging
from app.core.observe import RequestMetricsMiddleware
from app.core.observe_worker import (
@@ -70,6 +76,7 @@ from app.core.withdraw_reconcile_worker import (
start_withdraw_reconcile_worker,
stop_withdraw_reconcile_worker,
)
from app.integrations import meituan as mt_meituan
setup_logging(debug=settings.APP_DEBUG)
logger = logging.getLogger("shagua.main")
@@ -86,6 +93,9 @@ async def lifespan(_: FastAPI) -> AsyncIterator[None]:
settings.DATABASE_URL.split("://", 1)[0],
)
get_pricebot_client() # 预热透传 client:把建 SSL 上下文的一次性成本付在启动,首个领券请求即热
if settings.mt_cps_configured:
# 同理预热美团 CPS client(TLS 上下文 + 连接池建一次,后续 keep-alive 复用)
mt_meituan.get_client()
try:
# 预热离线地理库:首次加载 ~2.5M 行 CSV + 建 KDTree,摊到启动、不砸首个按城市过滤的请求
from app.utils import geo
@@ -98,6 +108,7 @@ async def lifespan(_: FastAPI) -> AsyncIterator[None]:
daily_exchange_task = start_daily_exchange_worker()
observe_task = start_observe_worker()
inactivity_task = start_inactivity_reset_worker()
llm_cost_backfill_task = start_llm_cost_backfill_worker()
try:
yield
finally:
@@ -107,7 +118,9 @@ async def lifespan(_: FastAPI) -> AsyncIterator[None]:
await stop_daily_exchange_worker(daily_exchange_task)
await stop_observe_worker(observe_task)
await stop_inactivity_reset_worker(inactivity_task)
await stop_llm_cost_backfill_worker(llm_cost_backfill_task)
await aclose_pricebot_client()
mt_meituan.close_client()
logger.info("shutting down")
@@ -141,6 +154,7 @@ app.include_router(auth_router)
app.include_router(user_router)
app.include_router(feedback_router)
app.include_router(analytics_router)
app.include_router(applog_router)
app.include_router(invite_router)
app.include_router(coupon_router)
app.include_router(device_router)
@@ -154,6 +168,8 @@ app.include_router(signin_router)
app.include_router(tasks_router)
app.include_router(savings_router)
app.include_router(ad_router)
# 新手引导视频(领券等候浮层前 N 次替代广告;运营后台传片,见 repositories/guide_video.py)
app.include_router(guide_video_router)
app.include_router(order_router)
app.include_router(report_router)
# 消息通知中心(PRD;数据落库 notification 表,见 repositories/notification.py)
+10 -7
View File
@@ -13,20 +13,22 @@ from app.models.analytics_selfstat import ( # noqa: F401
)
from app.models.app_config import AppConfig # noqa: F401
from app.models.comparison import ComparisonRecord # noqa: F401
from app.models.cps_activity import CpsActivity # noqa: F401
from app.models.cps_group import CpsGroup # noqa: F401
from app.models.cps_link import CpsClick, CpsLink # noqa: F401
from app.models.cps_order import CpsOrder # noqa: F401
from app.models.cps_wx_user import CpsWxUser # noqa: F401
from app.models.comparison_milestone import ComparisonMilestoneClaim # noqa: F401
from app.models.device import DeviceLiveness # noqa: F401
from app.models.coupon_state import ( # noqa: F401
CouponClaimEvent,
CouponClaimRecord,
CouponDailyCompletion,
CouponPromptEngagement,
CouponSession,
)
from app.models.cps_activity import CpsActivity # noqa: F401
from app.models.cps_group import CpsGroup # noqa: F401
from app.models.cps_link import CpsClick, CpsLink # noqa: F401
from app.models.cps_order import CpsOrder # noqa: F401
from app.models.cps_wx_user import CpsWxUser # noqa: F401
from app.models.device import DeviceLiveness # noqa: F401
from app.models.feedback import Feedback # noqa: F401
from app.models.guide_video import GuideVideoPlay # noqa: F401
from app.models.inactivity import ( # noqa: F401
InactivityNotificationLog,
InactivityResetLog,
@@ -37,11 +39,12 @@ from app.models.launch_confirm_sample import LaunchConfirmSample # noqa: F401
from app.models.meituan_coupon import MeituanCoupon # noqa: F401
from app.models.notification import Notification # noqa: F401
from app.models.onboarding import OnboardingCompletion # noqa: F401
from app.models.phone_rebind_log import PhoneRebindLog # noqa: F401
from app.models.ops_marquee_seed import OpsMarqueeSeed # noqa: F401
from app.models.ops_stat_config import OpsStatConfig # noqa: F401
from app.models.phone_rebind_log import PhoneRebindLog # noqa: F401
from app.models.price_observation import PriceObservation # noqa: F401
from app.models.price_report import PriceReport # noqa: F401
from app.models.risk import BehaviorEvent, RiskIncident, SubjectRestriction # noqa: F401
from app.models.savings import SavingsRecord # noqa: F401
from app.models.signin import SigninRecord # noqa: F401
from app.models.store_mapping import StoreMapping # noqa: F401
+9
View File
@@ -102,12 +102,21 @@ class ComparisonRecord(Base):
# done 帧 information 文案。成功:"在美团找到同店,到手价 ¥X…";
# 失败:具体原因(如"美团、京东外卖均未找到该商品")。前端在比价失败时当原因展示。
information: Mapped[str | None] = mapped_column(String(256), nullable=True)
# 失败卡「原因」行的展示文案(仅 status=failed 时非空):information 具体则=它;笼统则从
# platform_results 捞出的业务原因(打烊/未起送/找不到店或菜/单点不配送);纯系统失败为 None
# → 端侧显示品牌兜底「网络开小差…」。写路径(harvest_done / upsert_record)落库时派生。
# 见 repositories.comparison._derive_fail_display。
fail_reason: Mapped[str | None] = mapped_column(String(256), nullable=True)
# ===== 明细(JSON,越详细越好)=====
# 下单菜品 [{name, qty, specs?}]
items: Mapped[list] = mapped_column(_JSON, nullable=False, default=list)
# 逐平台对比 [{platform_id, platform_name, package, price, is_source, rank, coupon_saved, coupon_name, applied_coupons}](price/coupon_saved 单位:元,原样存;coupon_name=优惠来源名;applied_coupons=[{name,amount}] 多券明细)
comparison_results: Mapped[list] = mapped_column(_JSON, nullable=False, default=list)
# 展示模型统一数组(pricebot done.params.platforms 原样存): 每平台一行、自带
# status/is_best/display/display_order, 记录页据此直接渲染, 不再靠 comparison_results
# + 客户端合并 + 前端派生。老记录/旧客户端为空 → 前端回退老 comparison_results 渲染。
platforms: Mapped[list] = mapped_column(_JSON, nullable=False, default=list)
# 目标平台未找到、跳过的菜名
skipped_dish_names: Mapped[list] = mapped_column(_JSON, nullable=False, default=list)
# 客户端上报的原始 payload(calibration + done.params 全量),未来取数兜底
+34
View File
@@ -96,6 +96,40 @@ class CouponClaimRecord(Base):
)
class CouponClaimEvent(Base):
"""一次领券任务中的单券结果,按 ``(trace_id, coupon_id)`` 幂等。"""
__tablename__ = "coupon_claim_event"
__table_args__ = (
UniqueConstraint(
"trace_id", "coupon_id",
name="uq_coupon_claim_event_trace_coupon",
),
Index("ix_coupon_claim_event_date_env", "claim_date", "app_env"),
)
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
trace_id: Mapped[str] = mapped_column(String(64), nullable=False, index=True)
device_id: Mapped[str] = mapped_column(String(64), nullable=False)
user_id: Mapped[int | None] = mapped_column(Integer, index=True, nullable=True)
coupon_id: Mapped[str] = mapped_column(String(64), nullable=False)
claim_date: Mapped[date] = mapped_column(Date, nullable=False)
status: Mapped[str] = mapped_column(String(24), nullable=False)
app_env: Mapped[str | None] = mapped_column(String(16), index=True, nullable=True)
vendor: Mapped[str | None] = mapped_column(String(48), nullable=True)
coupon_name: Mapped[str | None] = mapped_column(String(128), nullable=True)
claimed_count: Mapped[int | None] = mapped_column(Integer, nullable=True)
reason: Mapped[str | None] = mapped_column(String(255), nullable=True)
extra: Mapped[dict | None] = mapped_column(_JSON, nullable=True)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), nullable=False
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), onupdate=func.now(),
nullable=False,
)
class CouponDailyCompletion(Base):
"""按 (device, 自然日) 记"今天是否已跑完整轮领券(到 done 帧)"——首页置灰源。
+73
View File
@@ -0,0 +1,73 @@
"""新手引导视频播放记录(领券浮层前 N 次用它替代广告)。
产品规则(2026-07 拍板):新用户点一键自动领取后的等候浮层,** 3 **不放广告,
改放运营后台上传的引导视频;每次固定 120 金币,中途关闭也算看完照发
口径:
- **计次按账号**(user_id),与设备无关 换设备不重新送 3
- **开播即计数**:客户端每次要展示浮层时调 `/api/v1/guide-video/start`,服务端当场
写一行(status='playing')并返回 play_token;`COUNT(*)` 即已用次数用户中途 kill
App 也算用掉一次(产品选定口径,防反复进出刷金币)
- **发币幂等** play_token 定位 + `status='playing'` 条件更新:并发两次上报只有一次
改到行(另一次 rowcount=0),所以只发一次币光有 play_token 唯一键挡不住 发币走的是
UPDATE, INSERT,撞不到任何唯一键
- **次数上限** (user_id, seq) 唯一键兜底,防并发 /start 绕过 COUNT 判定(见下)
与广告收益(ad_feed_reward_record)彻底分离:引导视频不是广告,不该进广告收益报表
"""
from __future__ import annotations
from datetime import datetime
from sqlalchemy import DateTime, ForeignKey, Index, Integer, String, UniqueConstraint, func
from sqlalchemy.orm import Mapped, mapped_column
from app.db.base import Base
class GuideVideoPlay(Base):
"""一次引导视频播放一行。开播时建(status='playing'),发币后置 'granted'"""
__tablename__ = "guide_video_play"
__table_args__ = (
# 客户端幂等键:同一次播放重复上报奖励只发一次。
UniqueConstraint("play_token", name="uq_guide_video_play_token"),
# 次数上限的**硬约束**:start_play 是无锁 check-then-insert(读 COUNT 算 seq 再插),
# N 个并发 /start 会都读到同一个已用次数、算出同一个 seq,不拦就能各拿一个 token、
# 各发一次金币,3 次上限形同虚设(改包即可无限刷)。seq 唯一 → 并发同 seq 必撞,
# start_play 捕获 IntegrityError 降级成"这次不放视频"。
# 用 unique Index 而非 UniqueConstraint:与迁移里的 create_index 对齐(SQLite 加约束
# 要整表重建),autogenerate 才不会每次报一条假 diff。
Index("uq_guide_video_play_user_seq", "user_id", "seq", unique=True),
)
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
user_id: Mapped[int] = mapped_column(
Integer, ForeignKey("user.id"), index=True, nullable=False
)
# 服务端生成下发给客户端的幂等键(uuid hex)。
play_token: Mapped[str] = mapped_column(String(64), nullable=False)
# 触发场景:目前只有 coupon(领券等候浮层);留字段以便日后比价等场景复用。
scene: Mapped[str] = mapped_column(String(16), nullable=False, default="coupon")
# 本账号第几次(1-based),= 建行时已有行数 + 1。日常判定仍以 COUNT 为准,但 (user_id, seq)
# 唯一键让并发 /start 只能成一个 —— 见 __table_args__。
seq: Mapped[int] = mapped_column(Integer, nullable=False, default=1)
# 当次下发的视频地址(运营换片后能回溯用户当时看的是哪支)。
video_url: Mapped[str | None] = mapped_column(String(512), nullable=True)
# 实发金币;未发时 0。
coin: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
# playing(已开播未发币) / granted(已发币)。
status: Mapped[str] = mapped_column(String(16), nullable=False, default="playing")
# 客户端上报时是否播完(true=自然播完 / false=中途关闭)。仅留痕:两者都发币。
completed: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
started_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), index=True, nullable=False
)
granted_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
def __repr__(self) -> str: # pragma: no cover
return (
f"<GuideVideoPlay user={self.user_id} seq={self.seq} "
f"{self.status} coin={self.coin}>"
)
+21 -1
View File
@@ -23,7 +23,7 @@ from __future__ import annotations
from datetime import datetime
from sqlalchemy import JSON, DateTime, Float, Integer, String, UniqueConstraint, func
from sqlalchemy import JSON, DateTime, Float, Index, Integer, String, UniqueConstraint, func
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import Mapped, mapped_column
@@ -95,3 +95,23 @@ class MeituanCoupon(Base):
f"<MeituanCoupon id={self.id} source={self.source} "
f"name={self.name!r} sale={self.sale_volume} comm={self.commission_percent}>"
)
# 首页「销量最高 / 智能推荐」两个 tab 的分页索引(见 api/v1/meituan.py 的 _paged_dedup_ids)。
# 两条 tab 都是 `WHERE city_id=? [+过滤] ORDER BY dedup_key, <排序键> DESC` 的 DISTINCT ON 去重,
# 列顺序对齐后 Postgres 可以顺着索引流式去重,免掉整城数据的排序 —— 否则每翻一页都要把该城
# 全部券重排一遍(下滑到底越来越慢的根因之一)。
# 定义放在类外:__table_args__ 里拿不到还没建好的类属性,写不了 .desc()。
Index(
"ix_meituan_coupon_city_dedup_sales",
MeituanCoupon.city_id,
MeituanCoupon.dedup_key,
MeituanCoupon.sale_volume_num.desc(),
MeituanCoupon.commission_percent.desc(),
)
Index(
"ix_meituan_coupon_city_dedup_comm",
MeituanCoupon.city_id,
MeituanCoupon.dedup_key,
MeituanCoupon.commission_percent.desc(),
)
+163
View File
@@ -0,0 +1,163 @@
"""通用行为事件、风险事件与主体限制。
这三张表不是风控监控页面专用表
- ``behavior_event`` 保存服务端权威的关键行为流水后续可复用于安全审计
漏斗核查和客诉排查
- ``risk_incident`` 保存规则命中后的可处置事件负责待处理/忽略/封禁/解除
生命周期
- ``subject_restriction`` 保存当前生效的主体限制统一承载设备账号等主体的
业务拦截状态
"""
from __future__ import annotations
from datetime import datetime
from sqlalchemy import (
JSON,
Boolean,
DateTime,
Index,
Integer,
String,
UniqueConstraint,
func,
true,
)
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import Mapped, mapped_column
from app.db.base import Base
_JSON = JSON().with_variant(JSONB(), "postgresql")
class BehaviorEvent(Base):
"""服务端权威行为流水;只增不改。"""
__tablename__ = "behavior_event"
__table_args__ = (
Index("ix_behavior_event_type_time", "event_type", "occurred_at"),
Index(
"ix_behavior_event_subject_time",
"subject_type",
"subject_id",
"occurred_at",
),
Index("ix_behavior_event_user_time", "user_id", "occurred_at"),
)
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
event_type: Mapped[str] = mapped_column(String(64), nullable=False)
subject_type: Mapped[str] = mapped_column(String(32), nullable=False)
subject_id: Mapped[str] = mapped_column(String(128), nullable=False)
user_id: Mapped[int | None] = mapped_column(Integer, nullable=True)
device_id: Mapped[str | None] = mapped_column(String(128), nullable=True)
device_model: Mapped[str | None] = mapped_column(String(128), nullable=True)
phone: Mapped[str | None] = mapped_column(String(20), nullable=True)
client_ip: Mapped[str | None] = mapped_column(String(64), nullable=True)
# success / failed / rejected / attempted
outcome: Mapped[str] = mapped_column(String(24), nullable=False, default="success")
reason: Mapped[str | None] = mapped_column(String(256), nullable=True)
details: Mapped[dict | None] = mapped_column(_JSON, nullable=True)
occurred_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, index=True
)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), nullable=False
)
class RiskIncident(Base):
"""规则命中后形成的可处置风险事件。"""
__tablename__ = "risk_incident"
__table_args__ = (
UniqueConstraint(
"rule_code",
"subject_type",
"subject_id",
"window_key",
name="uq_risk_incident_rule_subject_window",
),
Index("ix_risk_incident_rule_status", "rule_code", "status", "triggered_at"),
Index(
"ix_risk_incident_subject",
"subject_type",
"subject_id",
"triggered_at",
),
)
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
rule_code: Mapped[str] = mapped_column(String(64), nullable=False)
event_type: Mapped[str] = mapped_column(String(64), nullable=False)
subject_type: Mapped[str] = mapped_column(String(32), nullable=False)
subject_id: Mapped[str] = mapped_column(String(128), nullable=False)
window_key: Mapped[str] = mapped_column(String(64), nullable=False)
window_start: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
window_end: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
first_event_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
triggered_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
last_event_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
event_count: Mapped[int] = mapped_column(Integer, nullable=False)
# open / ignored / blocked / resolved
status: Mapped[str] = mapped_column(String(24), nullable=False, default="open")
action_reason: Mapped[str | None] = mapped_column(String(256), nullable=True)
handled_by: Mapped[int | None] = mapped_column(Integer, nullable=True)
handled_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
details: Mapped[dict | None] = mapped_column(_JSON, nullable=True)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), nullable=False
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
server_default=func.now(),
onupdate=func.now(),
nullable=False,
)
class SubjectRestriction(Base):
"""主体当前限制状态;同一主体同一作用域只有一条,可封禁后再解除/重启。"""
__tablename__ = "subject_restriction"
__table_args__ = (
UniqueConstraint(
"subject_type",
"subject_id",
"scope",
name="uq_subject_restriction_subject_scope",
),
Index(
"ix_subject_restriction_lookup",
"subject_type",
"subject_id",
"scope",
"active",
),
)
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
subject_type: Mapped[str] = mapped_column(String(32), nullable=False)
subject_id: Mapped[str] = mapped_column(String(128), nullable=False)
# auth_device / economic_account / all
scope: Mapped[str] = mapped_column(String(32), nullable=False)
active: Mapped[bool] = mapped_column(
Boolean, nullable=False, default=True, server_default=true()
)
reason: Mapped[str | None] = mapped_column(String(256), nullable=True)
incident_id: Mapped[int | None] = mapped_column(Integer, nullable=True)
created_by: Mapped[int | None] = mapped_column(Integer, nullable=True)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), nullable=False
)
revoked_by: Mapped[int | None] = mapped_column(Integer, nullable=True)
revoked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
details: Mapped[dict | None] = mapped_column(_JSON, nullable=True)
+8 -1
View File
@@ -11,7 +11,7 @@ from __future__ import annotations
from datetime import datetime, timezone
from sqlalchemy import Boolean, DateTime, Integer, String, false, func
from sqlalchemy import Boolean, DateTime, Integer, String, Text, false, func
from sqlalchemy.orm import Mapped, mapped_column
from app.db.base import Base
@@ -62,6 +62,13 @@ class User(Base):
Boolean, nullable=False, default=False, server_default=false()
)
# 运营人工风险标记。与自动风控分值分开:这里表达人工复核结论,并保留可编辑备注,
# 供邀请提现、其他提现和用户管理三个页面统一展示。
is_high_risk: Mapped[bool] = mapped_column(
Boolean, nullable=False, default=False, server_default=false(), index=True
)
high_risk_note: Mapped[str | None] = mapped_column(Text, nullable=True)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), nullable=False
)
-9
View File
@@ -96,15 +96,6 @@ class WithdrawOrder(Base):
"""
__tablename__ = "withdraw_order"
__table_args__ = (
Index(
"ux_withdraw_order_user_active",
"user_id",
unique=True,
sqlite_where=text("status IN ('reviewing', 'pending')"),
postgresql_where=text("status IN ('reviewing', 'pending')"),
),
)
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
user_id: Mapped[int] = mapped_column(
+304 -20
View File
@@ -5,7 +5,7 @@
"""
from __future__ import annotations
from datetime import datetime
from datetime import datetime, timedelta
from sqlalchemy import func, or_, select
from sqlalchemy.orm import Session, defer
@@ -14,8 +14,19 @@ from app.core.rewards import CN_TZ
from app.models.ad_feed_reward import AdFeedRewardRecord
from app.models.comparison import ComparisonRecord
from app.models.savings import SavingsRecord
from app.models.user import User
from app.schemas.compare_record import ComparisonRecordIn
DAILY_COMPARE_START_LIMIT = 100
class DailyCompareStartLimitExceeded(Exception):
"""The authenticated user has consumed today's comparison-start quota."""
class ComparisonTraceOwnershipError(Exception):
"""A trace id already belongs to a different authenticated user."""
def _yuan_to_cents(yuan: float | None) -> int | None:
"""元(float)→ 分(int)。None 透传。"""
@@ -41,16 +52,100 @@ def _product_names_from_items(items: list | None) -> str | None:
return joined[:500] or None
# ---- 失败记录的展示文案(记录页失败卡「原因」行)------------------------------
# information 具体就直出;笼统(_GENERIC_INFO)则从 platform_results 捞一条用户可读的业务
# 原因;捞不到 → None(端侧显示品牌兜底「网络开小差…」)。pricebot 把 store_closed /
# no_delivery 漏成了 status=failed,这里按 reason 关键字补判;打烊类 reason 常带一坨脏店名
# (店名+月售+起送+配送…),统一成简短模板。自动化黑话(搜索失败/读价失败/购物车残留/裸
# FAILED…)不给用户看 → 归入品牌兜底。
# pricebot 组不出具体原因时的笼统 information(线上统计的大头),一律走品牌兜底。
_GENERIC_INFO = {
"比价过程出错,请稍后重试",
"比价出错",
"比价未完成",
"done 参数缺少可验证的目标平台结果",
}
# 干净业务结局 status(直接可信),按展示优先级(越靠前越先选)。
_BIZ_STATUS_PRIORITY = (
"below_minimum",
"no_delivery",
"store_closed",
"items_not_found",
"store_not_found",
)
def _store_closed_text(reason: str | None) -> str:
"""打烊/暂停营业/休息类 reason 常带脏店名元数据 → 只留结论,套简短模板。"""
r = reason or ""
if "暂停营业" in r:
state = "暂停营业"
elif "休息" in r:
state = "休息中"
else:
state = "已打烊"
return f"门店{state},无法比价"
def _target_display_reason(platform_results: dict | None) -> str | None:
"""从逐平台结果里挑一条"可展示给用户"的失败原因;挑不到返回 None。
status 命中干净业务结局集 直接采信(打烊套模板,其余用 reason);
补判 pricebot 漏成 status=failed 的两类:打烊(套模板)单点不配送(reason 本身干净);
自动化黑话(搜索失败/读价失败/购物车残留/ FAILED)一律不展示 None"""
pr = platform_results or {}
targets = [
v for v in pr.values() if isinstance(v, dict) and not v.get("is_source")
]
for want in _BIZ_STATUS_PRIORITY: # ① 干净 status 优先
for v in targets:
if v.get("status") == want:
if want == "store_closed":
return _store_closed_text(v.get("reason"))
if v.get("reason"):
return v["reason"]
for v in targets: # ② 漏成 failed 的业务结局补判
if v.get("status") != "failed":
continue
reason = (v.get("reason") or "").strip()
if any(k in reason for k in ("打烊", "暂停营业", "休息")):
return _store_closed_text(reason)
if "单点不配送" in reason:
return reason
return None
def _derive_fail_display(
information: str | None, platform_results: dict | None
) -> str | None:
"""失败记录展示文案:information 具体则直出;笼统则从 platform_results 捞/补判;
都拿不到 None(端侧品牌兜底)仅在 status=failed 时调用"""
info = (information or "").strip()
text = info if (info and info not in _GENERIC_INFO) else _target_display_reason(
platform_results
)
return text[:256] if text else None
def _derive(payload: ComparisonRecordIn) -> dict:
"""从上报 payload 派生结构化列(best/saved/is_source_best/status)。"""
results = payload.comparison_results
_pr = payload.platform_results or {}
# 最优 = rank 最小的一条;协议已升序,但不信顺序,显式按 rank/price 兜底取最小价。
best = None
def _is_short(r) -> bool:
# 缺菜(漏菜)店: 少买了菜总价虚低, 不参与最优评选。逐平台 skipped 在 platform_results, 行里没有。
# platform_results 内层结构宽松(pricebot/老客户端透传, 可伪造), 值非 dict 时按"不缺菜"处理, 不崩。
info = _pr.get(r.platform_id) if r.platform_id else None
return isinstance(info, dict) and (info.get("skipped_dish_count") or 0) > 0
# 最优 = 非缺菜里 rank 最小(=最便宜)的一条;协议已升序,但不信顺序,显式按 rank/price 取。
# 源平台永远全菜, 故全目标缺菜时回落到源(is_source_best、saved=0), 不把虚低价当最低。
priced = [r for r in results if r.price is not None]
if priced:
clean = [r for r in priced if not _is_short(r)]
best = None
if clean:
best = min(
priced,
clean,
key=lambda r: (r.rank if r.rank is not None else 10**9, r.price),
)
@@ -69,8 +164,9 @@ def _derive(payload: ComparisonRecordIn) -> dict:
is_source_best = best.is_source if best is not None else None
# status:客户端显式给了就用;否则有"非源且有价"的结果=success,否则 failed
status = payload.status
# status:优先 pricebot record_status(区分 below_minimum/store_closed) → 客户端显式 status
# → 兜底"非源且有价"=success/否则 failed。record_status 让"未满起送"不再塌缩成 failed。
status = payload.record_status or payload.status
if status is None:
has_valid_target = any(
(not r.is_source) and r.price is not None for r in results
@@ -85,6 +181,11 @@ def _derive(payload: ComparisonRecordIn) -> dict:
"saved_amount_cents": saved_amount_cents,
"is_source_best": is_source_best,
"status": status,
"fail_reason": (
_derive_fail_display(payload.information, _pr)
if status == "failed"
else None
),
}
@@ -97,16 +198,35 @@ def upsert_record(
灰度期老客户端 POST /compare/record 走这条,与后端 harvest trace_id reconcile;
新客户端不再 POST(改由 compare.py 透传壳 harvest 落库)
"""
derived = _derive(payload)
# 单源派生: 与 harvest_done 一致, payload 带 platforms 时从它派生(唯一真相源
# _derive_from_platforms), 老客户端不带 platforms 时回退 _derive(从 comparison_results)。
if payload.platforms:
derived = _derive_from_platforms(payload.platforms, payload.record_status)
# 对齐 _derive 返回键(#189 fail_reason): 两路径 fields 键集一致, 覆盖已有行时不残留旧值
derived["fail_reason"] = (
_derive_fail_display(payload.information, payload.platform_results or {})
if derived["status"] == "failed"
else None
)
# 单源派生取自 platforms 源行(常无源平台元数据/店名)→ 空则用 payload 兜底不丢字段。
# 下面 fields 不再显式写这四个键, 统一由 derived 提供(否则 dict(store_name=..., **derived)
# 与 _derive_from_platforms 同名键撞键 TypeError)。
for _k in ("store_name", "source_platform_id", "source_platform_name", "source_package"):
if not derived.get(_k):
derived[_k] = getattr(payload, _k)
else:
derived = _derive(payload)
# _derive 只从 comparison_results 派生, 不含源平台四件套 / store_name → 从 payload 补,
# 与上面 platforms 分支键集对齐(fields 统一靠 **derived 提供这些列)。
for _k in ("store_name", "source_platform_id", "source_platform_name", "source_package"):
derived[_k] = getattr(payload, _k)
items = [it.model_dump(exclude_none=True) for it in payload.items]
fields = dict(
device_id=payload.device_id,
business_type=payload.business_type,
store_name=payload.store_name,
product_names=_product_names_from_items(items),
source_platform_id=payload.source_platform_id,
source_platform_name=payload.source_platform_name,
source_package=payload.source_package,
# store_name / source_platform_id / source_platform_name / source_package 统一由
# derived 提供(见上方两分支补齐), 不在此显式写 —— 否则与 _derive_from_platforms 撞键。
information=payload.information,
best_deeplink=payload.best_deeplink,
trace_url=payload.trace_url,
@@ -114,6 +234,7 @@ def upsert_record(
skipped_dish_count=payload.skipped_dish_count,
items=items,
comparison_results=[r.model_dump() for r in payload.comparison_results],
platforms=list(payload.platforms or []),
skipped_dish_names=list(payload.skipped_dish_names),
# 客户端环境 / 性能(debug,客户端上报;旧客户端为 None)
device_model=payload.device_model,
@@ -185,14 +306,30 @@ def upsert_record(
# ============================================================
def _derive_from_results(results: list[dict]) -> dict:
def _derive_from_results(
results: list[dict], platform_results: dict | None = None,
record_status: str | None = None,
) -> dict:
"""从 done 帧 comparison_results(pricebot 原始 dict 列表)派生结构化列。
等价 _derive,但吃原始字段(is_source/price/rank/platform_id/store_name...)而非 pydantic 对象"""
等价 _derive,但吃原始字段(is_source/price/rank/platform_id/store_name...)而非 pydantic 对象
platform_results(done.params.platform_results): 逐平台 skipped_dish_count 在这里(行里没有)
传入则派生 best 时排除缺菜(漏菜) 少买了菜总价虚低, 不能当记录级"最低价"/算虚假省额;
源平台永远全菜, 故全目标缺菜时 best 回落到源(is_source_best不虚报省)不传 rank/price, 行为不变"""
_pr = platform_results or {}
def _is_short(r: dict) -> bool:
# platform_results 内层结构宽松(pricebot/客户端透传), 值非 dict 时按"不缺菜"处理, 不崩。
pid = r.get("platform_id")
info = _pr.get(pid) if pid else None
return isinstance(info, dict) and (info.get("skipped_dish_count") or 0) > 0
priced = [r for r in results if r.get("price") is not None]
clean = [r for r in priced if not _is_short(r)] # 缺菜店排除出最优评选
best = None
if priced:
if clean:
best = min(
priced,
clean,
key=lambda r: (r.get("rank") if r.get("rank") is not None else 10**9, r["price"]),
)
src_row = next((r for r in results if r.get("is_source")), None)
@@ -219,7 +356,53 @@ def _derive_from_results(results: list[dict]) -> dict:
"saved_amount_cents": saved_amount_cents,
"is_source_best": best.get("is_source") if best else None,
"store_name": (src_row or {}).get("store_name") or None,
"status": "success" if has_valid_target else "failed",
# 记录级结局: 优先用 pricebot 下发的 record_status(区分 below_minimum/store_closed,
# 不再把"未满起送"塌缩成 failed → 记录页不再误报"网络开小差"); 旧 pricebot 未下发时
# 回退老的 success/failed 二态派生, 向后兼容。
"status": record_status or ("success" if has_valid_target else "failed"),
}
def _derive_from_platforms(
platforms: list, record_status: str | None = None,
) -> dict:
"""从 done 帧 platforms(每平台一行、渲染就绪)派生结构化列——**单一真相源**。
best_* 直接取 platforms is_best 的那一行source_* role=source ,与前端读的
platforms 天然一致(不再像 _derive_from_results 那样从 comparison_results 二次评最优,
消除"标量列 vs platforms"双源不一致)platforms 非空时优先走这里; pricebot
platforms 时调用方回退 _derive_from_results(向后兼容)"""
rows = [p for p in (platforms or []) if isinstance(p, dict)]
src = next((p for p in rows if p.get("role") == "source"), None)
best = next((p for p in rows if p.get("is_best")), None)
source_price_cents = _yuan_to_cents(src.get("price")) if src else None
best_price_cents = _yuan_to_cents(best.get("price")) if best else None
saved_amount_cents = None
if source_price_cents is not None and best_price_cents is not None:
saved_amount_cents = source_price_cents - best_price_cents
has_valid_target = any(
p.get("role") != "source" and p.get("price") is not None for p in rows
)
# store_name: 优先源行; recompare 场景源平台自己当目标、源行被目标覆盖(pricebot
# _build_platform_rows 有意去重, platforms 无 role=source 行)→ 回退 best 行 → 首个有店名
# 的行(显示现场实际比到的店), 免得记录页店名空掉兜底显示成"比价"。正常比价有源行不走回退。
store_name = (
(src or {}).get("store_name")
or (best or {}).get("store_name")
or next((p.get("store_name") for p in rows if p.get("store_name")), None)
)
return {
"source_platform_id": (src or {}).get("platform_id"),
"source_platform_name": (src or {}).get("platform_name"),
"source_package": (src or {}).get("package"),
"source_price_cents": source_price_cents,
"best_platform_id": (best or {}).get("platform_id"),
"best_platform_name": (best or {}).get("platform_name"),
"best_price_cents": best_price_cents,
"saved_amount_cents": saved_amount_cents,
"is_source_best": (best.get("role") == "source") if best else None,
"store_name": store_name or None,
"status": record_status or ("success" if has_valid_target else "failed"),
}
@@ -243,6 +426,78 @@ def _get_by_trace(db: Session, trace_id: str) -> ComparisonRecord | None:
).scalar_one_or_none()
def reserve_daily_start(
db: Session,
*,
user_id: int,
trace_id: str,
business_type: str = "food",
device_id: str | None = None,
now: datetime | None = None,
) -> tuple[ComparisonRecord, int]:
"""Atomically reserve one of a user's 100 Beijing-day comparison starts.
``trace_id`` makes client retries idempotent. Locking the user row serializes
concurrent starts for one account, so parallel requests cannot both consume
the final available slot. The reservation is the existing ``running``
comparison row; later result reporting updates that same row.
"""
db.execute(select(User.id).where(User.id == user_id).with_for_update()).scalar_one()
existing = _get_by_trace(db, trace_id)
if existing is not None:
if existing.user_id not in (None, user_id):
raise ComparisonTraceOwnershipError
if existing.user_id is None:
existing.user_id = user_id
if existing.device_id is None and device_id:
existing.device_id = device_id
db.commit()
db.refresh(existing)
existing_at = existing.created_at
if existing_at.tzinfo is not None:
existing_at = existing_at.astimezone(CN_TZ).replace(tzinfo=None)
day_start = existing_at.replace(hour=0, minute=0, second=0, microsecond=0)
day_end = day_start + timedelta(days=1)
used = db.scalar(
select(func.count(ComparisonRecord.id)).where(
ComparisonRecord.user_id == user_id,
ComparisonRecord.created_at >= day_start,
ComparisonRecord.created_at < day_end,
)
) or 0
return existing, int(used)
current = now or datetime.now(CN_TZ)
if current.tzinfo is not None:
current = current.astimezone(CN_TZ).replace(tzinfo=None)
day_start = current.replace(hour=0, minute=0, second=0, microsecond=0)
day_end = day_start + timedelta(days=1)
used = db.scalar(
select(func.count(ComparisonRecord.id)).where(
ComparisonRecord.user_id == user_id,
ComparisonRecord.created_at >= day_start,
ComparisonRecord.created_at < day_end,
)
) or 0
if used >= DAILY_COMPARE_START_LIMIT:
raise DailyCompareStartLimitExceeded
rec = ComparisonRecord(
trace_id=trace_id,
user_id=user_id,
business_type=business_type or "food",
device_id=device_id,
status="running",
created_at=current,
)
db.add(rec)
db.commit()
db.refresh(rec)
return rec, int(used) + 1
def harvest_running(
db: Session,
*,
@@ -306,12 +561,40 @@ def harvest_done(
返回 (记录, 是否本次****落成 success)供调用方据此幂等发一次邀请奖
行不存在(理论上帧0已建;防御)则新建"""
results = done_params.get("comparison_results") or []
derived = _derive_from_results(results)
# 菜品:pricebot 已把源单菜品塞进 comparison_results[源行].items
items = next((r.get("items") or [] for r in results if r.get("is_source")), [])
# 展示模型统一数组(pricebot 新增, 每平台一行自带 status/is_best): 原样存, 记录页据此直渲染。
# record_status: 记录级结局(success/below_minimum/store_closed/failed), 覆盖老二态派生。
platforms = done_params.get("platforms") or []
record_status = done_params.get("record_status")
# 单源派生: platforms(含 pricebot 权威 is_best)是唯一真相源, best_*/source_*/saved/status
# 全从它取 → 与前端读的 platforms 天然一致; 菜品也取 platforms 源行。老 pricebot 无
# platforms 时回退从 comparison_results 派生(向后兼容)。
if platforms:
derived = _derive_from_platforms(platforms, record_status)
# 菜品优先源行; recompare 无源行 → 回退 best 行 → 首个有菜品的行(同 store_name 回退)
_item_row = (
next((p for p in platforms if isinstance(p, dict) and p.get("role") == "source"), None)
or next((p for p in platforms if isinstance(p, dict) and p.get("is_best")), None)
or next((p for p in platforms if isinstance(p, dict) and p.get("items")), None)
)
items = (_item_row or {}).get("items") or []
else:
derived = _derive_from_results(
results, done_params.get("platform_results"), record_status
)
# pricebot 已把源单菜品塞进 comparison_results[源行].items
items = next((r.get("items") or [] for r in results if r.get("is_source")), [])
# 失败展示原因(#189): platforms / results 两个派生分支的 status 都可能 failed, 统一在此算
fail_reason = (
_derive_fail_display(
done_params.get("information"), done_params.get("platform_results")
)
if derived["status"] == "failed"
else None
)
fields = dict(
business_type=business_type or "food",
information=done_params.get("information") or None,
fail_reason=fail_reason,
# best_deeplink 来自客户端剪贴板采集,harvest 拿不到 → 留空(灰度期 fromComparison 会补;
# 纯 harvest 行「再次比价」退化为按 package 拉起 App。要精确深链需客户端另传,后续)。
trace_url=trace_url or done_params.get("trace_url"),
@@ -319,6 +602,7 @@ def harvest_done(
skipped_dish_count=done_params.get("skipped_dish_count"),
skipped_dish_names=list(done_params.get("skipped_dish_names") or []),
comparison_results=results,
platforms=platforms,
items=items,
product_names=_product_names_from_items(items),
raw_payload=done_params,
+39 -2
View File
@@ -14,6 +14,7 @@ from sqlalchemy.exc import IntegrityError
from sqlalchemy.orm import Session
from app.models.coupon_state import (
CouponClaimEvent,
CouponClaimRecord,
CouponDailyCompletion,
CouponPromptEngagement,
@@ -164,11 +165,12 @@ def record_claims(
results: list[dict],
app_env: str | None = None,
) -> int:
"""一批券领取结果幂等写入,返回写入(新增 + 更新)条数
"""一批券领取结果同时写入每日资产表和逐次事件表
results 单项取自 pricebot last_coupon_result / done.coupon_results,识别字段:
coupon_id(必需)/ status(必需)/ name / vendor / reason /(display_count)
(device, coupon_id, 今天) 唯一:重复上报同张券走更新(status 以最后一次为准)
- CouponClaimRecord (device, coupon_id, 今天) 幂等供每日资产口径使用
- CouponClaimEvent (trace_id, coupon_id) 幂等 admin 逐场统计使用
"""
today = today_cn()
written = 0
@@ -208,6 +210,41 @@ def record_claims(
claimed_count=count, trace_id=trace_id, reason=r.get("reason"),
extra=r,
))
if trace_id:
event = db.execute(
select(CouponClaimEvent).where(
CouponClaimEvent.trace_id == trace_id,
CouponClaimEvent.coupon_id == coupon_id,
)
).scalar_one_or_none()
if event is not None:
event.device_id = device_id
event.status = status
event.reason = r.get("reason")
event.vendor = r.get("vendor")
event.coupon_name = r.get("name")
event.extra = r
if user_id is not None:
event.user_id = user_id
if count is not None:
event.claimed_count = count
if app_env is not None:
event.app_env = app_env
else:
db.add(CouponClaimEvent(
trace_id=trace_id,
device_id=device_id,
user_id=user_id,
coupon_id=coupon_id,
claim_date=today,
status=status,
app_env=app_env,
vendor=r.get("vendor"),
coupon_name=r.get("name"),
claimed_count=count,
reason=r.get("reason"),
extra=r,
))
written += 1
if written == 0:
return 0
+290
View File
@@ -0,0 +1,290 @@
"""新手引导视频:运营配置读写 + 播放计次 + 发币。
**配置**(开关 / 视频地址 / 前几次 / 每次金币)整体作为一个 JSON 存进通用 app_config
(key=coupon_guide_video),写法完全对齐 feedback_qr 不进 CONFIG_DEFS,所以不会污染
系统配置页的通用列表,由本模块独占维护
**计次**按账号(user_id)**开播即计数**:客户端每次要展示领券等候浮层时调
`/api/v1/guide-video/start`,命中则当场写一行 guide_video_play(status='playing')
已用次数 = 该账号的行数,达到 max_plays 后不再下发,客户端改放广告(原逻辑)
COUNT 判定本身无锁,真正卡住次数上限的是 (user_id, seq) 唯一键:并发 /start 只能成一个
**发币**幂等键是 play_token,落地方式是 `status='playing' 'granted'` **条件更新**:
同一次播放重复上报只入账一次(网络重试 / 关闭与播完同时触发都靠它挡住)
中途关闭也照发 产品拍板中途关闭也算看完
两处都是直接铸币的路径,改动前先看 `start_play` / `grant_play` 上的并发注释
"""
from __future__ import annotations
import uuid
from datetime import datetime
from typing import Any
from sqlalchemy import func, select, update
from sqlalchemy.exc import IntegrityError
from sqlalchemy.orm import Session
from app.core import rewards
from app.models.app_config import AppConfig
from app.models.guide_video import GuideVideoPlay
from app.repositories import wallet as crud_wallet
_KEY = "coupon_guide_video"
#: 金币流水 biz_type。客户端收益明细按它显示「新手引导视频奖励」。
BIZ_TYPE = "guide_video"
# 默认值 = 「运营还没配」时的行为:video_url 为空 → 一律不下发引导视频,浮层维持现状(放广告)。
# 所以本功能上线后**不配视频就等于没上线**,不会影响存量用户。
_DEFAULTS: dict[str, Any] = {
"enabled": True,
"video_url": None, # None/空 = 未配片 → 不下发,浮层照旧放广告
"max_plays": 3, # 每个账号前 N 次浮层放引导视频
"reward_coin": 120, # 每次固定金币
}
_FIELDS = tuple(_DEFAULTS.keys())
# 后台可配范围的护栏:防手滑把次数/金币填成天文数字(配置直接决定发币)。
MAX_PLAYS_LIMIT = 50
REWARD_COIN_LIMIT = 10_000
# ===== 配置 =====
def _merge(raw: Any) -> dict[str, Any]:
"""DB 里(可能不全的)dict 叠加到默认上,得到完整配置(4 个字段,无 updated_at)。"""
out = dict(_DEFAULTS)
if isinstance(raw, dict):
for k in _FIELDS:
v = raw.get(k)
if v is not None:
out[k] = v
return out
def get_config(db: Session) -> dict[str, Any]:
"""完整配置 + updated_at(admin 读 / 业务读共用)。"""
row = db.get(AppConfig, _KEY)
cfg = _merge(row.value if row is not None else None)
cfg["updated_at"] = row.updated_at.isoformat() if row is not None and row.updated_at else None
return cfg
def _write(db: Session, value: dict[str, Any], *, admin_id: int, commit: bool) -> dict[str, Any]:
"""整体覆写该行(value 须为完整字段 dict),返回合并后的完整配置(含 updated_at)。"""
row = db.get(AppConfig, _KEY)
if row is None:
row = AppConfig(key=_KEY, value=value, updated_by_admin_id=admin_id)
db.add(row)
else:
row.value = value # 整体重新赋值,SQLAlchemy 才侦测得到变更
row.updated_by_admin_id = admin_id
if commit:
db.commit()
db.refresh(row)
else:
db.flush()
out = _merge(row.value)
out["updated_at"] = row.updated_at.isoformat() if row.updated_at else None
return out
def update_config(
db: Session,
*,
enabled: bool | None = None,
max_plays: int | None = None,
reward_coin: int | None = None,
admin_id: int,
commit: bool = True,
) -> tuple[dict[str, Any], dict[str, Any]]:
"""改开关 / 次数 / 金币(只改传了的字段;视频走 set_video)。返回 (before, after) 供审计。"""
row = db.get(AppConfig, _KEY)
before = _merge(row.value if row is not None else None)
new_value = {k: before[k] for k in _FIELDS}
if enabled is not None:
new_value["enabled"] = enabled
if max_plays is not None:
new_value["max_plays"] = max(0, min(int(max_plays), MAX_PLAYS_LIMIT))
if reward_coin is not None:
new_value["reward_coin"] = max(0, min(int(reward_coin), REWARD_COIN_LIMIT))
after = _write(db, new_value, admin_id=admin_id, commit=commit)
return before, after
def set_video(
db: Session, video_url: str | None, *, admin_id: int, commit: bool = True
) -> tuple[dict[str, Any], dict[str, Any]]:
"""设置/清空引导视频地址。返回 (before, after);before['video_url'] 供调用方删旧文件。"""
row = db.get(AppConfig, _KEY)
before = _merge(row.value if row is not None else None)
new_value = {k: before[k] for k in _FIELDS}
new_value["video_url"] = video_url
after = _write(db, new_value, admin_id=admin_id, commit=commit)
return before, after
# ===== 播放计次 =====
def used_plays(db: Session, user_id: int) -> int:
"""该账号已用掉的引导视频次数(开播即算,含未发币的)。"""
return int(
db.execute(
select(func.count()).select_from(GuideVideoPlay).where(
GuideVideoPlay.user_id == user_id
)
).scalar_one()
)
def play_stats(db: Session) -> dict[str, int]:
"""全站播放统计(admin 页展示):总播放次数 / 其中已发币次数。"""
total = int(
db.execute(select(func.count()).select_from(GuideVideoPlay)).scalar_one()
)
granted = int(
db.execute(
select(func.count()).select_from(GuideVideoPlay).where(
GuideVideoPlay.status == "granted"
)
).scalar_one()
)
return {"total_plays": total, "granted_plays": granted}
def start_play(
db: Session, user_id: int, *, scene: str = "coupon", commit: bool = True
) -> dict[str, Any]:
"""决定这次浮层是否放引导视频;命中则**当场计次**并返回 play_token。
返回 dict:
should_play 是否放引导视频(False 客户端照旧放广告)
video_url 相对地址(/media/...);客户端自行拼 BASE_URL
play_token 发币幂等键(should_play=False 时为空串)
reward_coin 播完/中途关闭都发的固定金币
seq / remaining 第几次 / 发完这次还剩几次(仅展示与排查用)
"""
cfg = get_config(db)
video_url = (cfg.get("video_url") or "").strip()
max_plays = int(cfg.get("max_plays") or 0)
reward_coin = int(cfg.get("reward_coin") or 0)
used = used_plays(db, user_id)
def _miss(used_now: int) -> dict[str, Any]:
return {
"should_play": False,
"video_url": None,
"play_token": "",
"reward_coin": reward_coin,
"seq": used_now,
"remaining": max(0, max_plays - used_now),
}
if not cfg.get("enabled") or not video_url or max_plays <= 0 or used >= max_plays:
return _miss(used)
seq = used + 1
play = GuideVideoPlay(
user_id=user_id,
play_token=uuid.uuid4().hex,
scene=scene,
seq=seq,
video_url=video_url,
coin=0,
status="playing",
completed=0,
started_at=datetime.now(rewards.CN_TZ).replace(tzinfo=None),
)
db.add(play)
# 上面的 COUNT 判定是无锁 check-then-insert:并发 /start 会都算出同一个 seq。
# (user_id, seq) 唯一键让只有一个能落库,其余撞键 → 回滚后按"这次不放视频"降级,
# 客户端照旧走广告链路。没有它,并发就能绕过 max_plays 无限刷金币。
try:
if commit:
db.commit()
else:
db.flush()
except IntegrityError:
db.rollback()
return _miss(used_plays(db, user_id))
return {
"should_play": True,
"video_url": video_url,
"play_token": play.play_token,
"reward_coin": reward_coin,
"seq": seq,
"remaining": max(0, max_plays - seq),
}
def _find_play(db: Session, user_id: int, token: str) -> GuideVideoPlay | None:
"""按 (play_token, user_id) 取播放行 —— 带 user_id 是防拿别人的 token 来兑。"""
return db.execute(
select(GuideVideoPlay).where(
GuideVideoPlay.play_token == token,
GuideVideoPlay.user_id == user_id,
)
).scalar_one_or_none()
def grant_play(
db: Session, user_id: int, *, play_token: str, completed: bool
) -> dict[str, Any]:
"""按 play_token 发这次引导视频的金币(幂等)。播完 / 中途关闭都发。
返回 {granted, coin, status}:granted=True 表示**本次调用真的入账了**;
重复上报返回 granted=False + 已发金币(客户端据此不重复累加 toast 金额)
"""
token = (play_token or "").strip()
# 金币额度以**服务端配置**为准,不信客户端(客户端只上报"播完/关闭")。
coin = int(get_config(db).get("reward_coin") or 0)
# 幂等核心:把 status 放进 WHERE 做条件更新(compare-and-set),而不是"先读再判再写"。
# 「播完」与「✕ 关闭」抢跑、或客户端超时重试时,两个请求会都读到 status='playing',
# 无锁的话就都往下发币、都 commit,金币入账两次(不用恶意,重试就会中招)。改成条件更新后
# 并发里只有一条 rowcount=1,另一条拿 0 → 按已发返回,不二次铸币。
# (PG READ COMMITTED 下后到的 UPDATE 阻塞到对手提交,再按新版本重判 status;SQLite 写串行。)
#
# 别指望 IntegrityError 兜底:这里只 UPDATE 不 INSERT,撞不到 uq_guide_video_play_token;
# 而 biz_type='guide_video' 的金币流水也不在 ux_coin_transaction_task_ref 的谓词
# (biz_type LIKE 'task%')覆盖范围内 —— 两个唯一键在这条路径上都是不生效的。
won = db.execute(
update(GuideVideoPlay)
.where(
GuideVideoPlay.play_token == token,
GuideVideoPlay.user_id == user_id,
GuideVideoPlay.status == "playing",
)
.values(
status="granted",
coin=coin,
completed=1 if completed else 0,
granted_at=datetime.now(rewards.CN_TZ).replace(tzinfo=None),
)
.execution_options(synchronize_session=False)
).rowcount
if not won:
# 没抢到:token 不存在 / 不是本人的 / 已被另一次上报发过。回滚拿干净快照再区分两者
# (对手此时必然已提交 —— 我们就是被它挡下的,所以读得到它写的 coin)。
db.rollback()
play = _find_play(db, user_id, token)
if play is None:
return {"granted": False, "coin": 0, "status": "not_found"}
return {"granted": False, "coin": play.coin, "status": "already_granted"}
if coin > 0:
crud_wallet.grant_coins(
db,
user_id,
coin,
biz_type=BIZ_TYPE,
ref_id=token,
remark="新手引导视频奖励",
)
db.commit()
return {"granted": True, "coin": coin, "status": "granted"}
+569
View File
@@ -0,0 +1,569 @@
"""通用行为事件、风险规则与主体限制仓库。"""
from __future__ import annotations
from dataclasses import dataclass
from datetime import UTC, datetime, timedelta
from zoneinfo import ZoneInfo
from sqlalchemy import func, select
from sqlalchemy.exc import IntegrityError
from sqlalchemy.orm import Session
from app.core.config_schema import (
RISK_COMPARE_DAILY_THRESHOLD_KEY,
RISK_ONECLICK_DAILY_THRESHOLD_KEY,
RISK_SMS_HOURLY_THRESHOLD_KEY,
)
from app.models.app_config import AppConfig
from app.models.comparison import ComparisonRecord
from app.models.risk import BehaviorEvent, RiskIncident, SubjectRestriction
from app.repositories import app_config
CN_TZ = ZoneInfo("Asia/Shanghai")
RULE_SMS_HOURLY = "sms_device_hourly_5"
RULE_ONECLICK_DAILY = "oneclick_device_daily_20"
RULE_COMPARE_DAILY = "compare_account_daily_100"
EVENT_SMS_SEND = "sms_send"
EVENT_SMS_LOGIN = "sms_login"
EVENT_ONECLICK_LOGIN = "oneclick_login"
SCOPE_AUTH_DEVICE = "auth_device"
SCOPE_ECONOMIC_ACCOUNT = "economic_account"
AUTO_RESOLVED_REASON = "规则阈值调整后不再命中"
MANUAL_RESET_REASON = "管理员重置报警计数"
RISK_RESET_BASELINES_KEY = "risk_monitor_reset_baselines"
@dataclass(frozen=True)
class RuleSpec:
code: str
event_type: str
subject_type: str
threshold_key: str
window: str
count_outcomes: tuple[str, ...]
RULES: dict[str, RuleSpec] = {
RULE_SMS_HOURLY: RuleSpec(
code=RULE_SMS_HOURLY,
event_type=EVENT_SMS_SEND,
subject_type="device",
threshold_key=RISK_SMS_HOURLY_THRESHOLD_KEY,
window="hour",
count_outcomes=("success",),
),
RULE_ONECLICK_DAILY: RuleSpec(
code=RULE_ONECLICK_DAILY,
event_type=EVENT_ONECLICK_LOGIN,
subject_type="device",
threshold_key=RISK_ONECLICK_DAILY_THRESHOLD_KEY,
window="day",
count_outcomes=("success", "failed"),
),
}
RULE_THRESHOLD_KEYS: dict[str, str] = {
RULE_SMS_HOURLY: RISK_SMS_HOURLY_THRESHOLD_KEY,
RULE_ONECLICK_DAILY: RISK_ONECLICK_DAILY_THRESHOLD_KEY,
RULE_COMPARE_DAILY: RISK_COMPARE_DAILY_THRESHOLD_KEY,
}
def get_rule_threshold(db: Session, rule_code: str) -> int:
"""读取规则当前阈值;配置表为空时回退上线前的 5/20/100 默认值。"""
return int(app_config.get_value(db, RULE_THRESHOLD_KEYS[rule_code]))
def utcnow() -> datetime:
return datetime.now(UTC)
def _ensure_aware(value: datetime) -> datetime:
return value.replace(tzinfo=UTC) if value.tzinfo is None else value
def get_rule_reset_at(db: Session, rule_code: str) -> datetime | None:
"""返回规则最近一次全局重置时间;异常旧值按未重置处理。"""
row = db.get(AppConfig, RISK_RESET_BASELINES_KEY)
if row is None or not isinstance(row.value, dict):
return None
raw = row.value.get(rule_code)
if not isinstance(raw, str):
return None
try:
parsed = datetime.fromisoformat(raw.replace("Z", "+00:00"))
except ValueError:
return None
return _ensure_aware(parsed).astimezone(UTC)
def reset_rule_baselines(
db: Session,
*,
rule_codes: tuple[str, ...],
reset_at: datetime,
admin_id: int,
) -> None:
"""持久化全局重置基线;不删除任何行为或比价流水。"""
row = db.get(AppConfig, RISK_RESET_BASELINES_KEY)
value = dict(row.value) if row and isinstance(row.value, dict) else {}
timestamp = _ensure_aware(reset_at).astimezone(UTC).isoformat()
value.update({rule_code: timestamp for rule_code in rule_codes})
if row is None:
db.add(
AppConfig(
key=RISK_RESET_BASELINES_KEY,
value=value,
updated_by_admin_id=admin_id,
)
)
else:
row.value = value
row.updated_by_admin_id = admin_id
db.flush()
def _window_bounds(at: datetime, window: str) -> tuple[str, datetime, datetime]:
local = _ensure_aware(at).astimezone(CN_TZ)
if window == "hour":
start_local = local.replace(minute=0, second=0, microsecond=0)
key = start_local.strftime("%Y-%m-%dT%H")
end_local = start_local + timedelta(hours=1)
elif window == "day":
start_local = local.replace(hour=0, minute=0, second=0, microsecond=0)
key = start_local.strftime("%Y-%m-%d")
end_local = start_local + timedelta(days=1)
else: # pragma: no cover - 规则声明错误应尽早暴露
raise ValueError(f"unsupported risk window: {window}")
return key, start_local.astimezone(UTC), end_local.astimezone(UTC)
def _event_stats(
db: Session,
spec: RuleSpec,
*,
subject_id: str,
start: datetime,
end: datetime,
threshold: int,
) -> tuple[int, datetime | None, datetime | None, datetime | None]:
filters = (
BehaviorEvent.event_type == spec.event_type,
BehaviorEvent.subject_type == spec.subject_type,
BehaviorEvent.subject_id == subject_id,
BehaviorEvent.outcome.in_(spec.count_outcomes),
BehaviorEvent.occurred_at >= start,
BehaviorEvent.occurred_at < end,
)
count, first_at, last_at = db.execute(
select(
func.count(BehaviorEvent.id),
func.min(BehaviorEvent.occurred_at),
func.max(BehaviorEvent.occurred_at),
).where(*filters)
).one()
triggered_at = None
if int(count or 0) >= threshold:
triggered_at = db.execute(
select(BehaviorEvent.occurred_at)
.where(*filters)
.order_by(BehaviorEvent.occurred_at.asc(), BehaviorEvent.id.asc())
.offset(threshold - 1)
.limit(1)
).scalar_one()
return int(count or 0), first_at, last_at, triggered_at
def _upsert_incident(
db: Session,
*,
rule_code: str,
event_type: str,
subject_type: str,
subject_id: str,
window_key: str,
window_start: datetime,
window_end: datetime,
first_event_at: datetime,
triggered_at: datetime,
last_event_at: datetime,
event_count: int,
details: dict | None = None,
) -> RiskIncident:
incident = db.execute(
select(RiskIncident).where(
RiskIncident.rule_code == rule_code,
RiskIncident.subject_type == subject_type,
RiskIncident.subject_id == subject_id,
RiskIncident.window_key == window_key,
)
).scalar_one_or_none()
if incident is None:
incident = RiskIncident(
rule_code=rule_code,
event_type=event_type,
subject_type=subject_type,
subject_id=subject_id,
window_key=window_key,
window_start=window_start,
window_end=window_end,
first_event_at=first_event_at,
triggered_at=triggered_at,
last_event_at=last_event_at,
event_count=event_count,
status="open",
details=details,
)
# 并发首次命中可能同时插入;保存点只回滚重复 incident,不丢行为流水。
try:
with db.begin_nested():
db.add(incident)
db.flush()
except IntegrityError:
incident = db.execute(
select(RiskIncident).where(
RiskIncident.rule_code == rule_code,
RiskIncident.subject_type == subject_type,
RiskIncident.subject_id == subject_id,
RiskIncident.window_key == window_key,
)
).scalar_one()
else:
reopened = incident.status == "resolved" and incident.action_reason in (
AUTO_RESOLVED_REASON,
MANUAL_RESET_REASON,
)
if reopened:
incident.status = "open"
incident.action_reason = None
incident.handled_by = None
incident.handled_at = None
# 同一自然窗口内重置后会复用唯一 incident;明细窗口必须同步切到新基线,
# 否则展开时会把重置前的旧流水也混进来。
incident.window_start = window_start
incident.window_end = window_end
incident.first_event_at = first_event_at
if incident.status == "open":
incident.triggered_at = triggered_at
incident.last_event_at = last_event_at
incident.event_count = event_count
# 已忽略/封禁的事件只更新事实数据,不重新打开。
if details:
incident.details = {**(incident.details or {}), **details}
return incident
def evaluate_behavior_rule(
db: Session, *, rule_code: str, subject_id: str, at: datetime
) -> RiskIncident | None:
spec = RULES[rule_code]
threshold = get_rule_threshold(db, rule_code)
window_key, window_start, end = _window_bounds(at, spec.window)
reset_at = get_rule_reset_at(db, rule_code)
start = max(window_start, reset_at) if reset_at else window_start
count, first_at, last_at, triggered_at = _event_stats(
db,
spec,
subject_id=subject_id,
start=start,
end=end,
threshold=threshold,
)
if count < threshold or first_at is None or last_at is None or triggered_at is None:
return None
return _upsert_incident(
db,
rule_code=spec.code,
event_type=spec.event_type,
subject_type=spec.subject_type,
subject_id=subject_id,
window_key=window_key,
window_start=start,
window_end=end,
first_event_at=first_at,
triggered_at=triggered_at,
last_event_at=last_at,
event_count=count,
)
def reconcile_behavior_rule(
db: Session,
*,
rule_code: str,
at: datetime | None = None,
commit: bool = True,
) -> int:
"""按当前阈值重算短信/一键登录当前窗口,并收起已不再命中的待处理告警。"""
spec = RULES[rule_code]
current = at or utcnow()
threshold = get_rule_threshold(db, rule_code)
window_key, window_start, end = _window_bounds(current, spec.window)
reset_at = get_rule_reset_at(db, rule_code)
start = max(window_start, reset_at) if reset_at else window_start
filters = (
BehaviorEvent.event_type == spec.event_type,
BehaviorEvent.subject_type == spec.subject_type,
BehaviorEvent.outcome.in_(spec.count_outcomes),
BehaviorEvent.occurred_at >= start,
BehaviorEvent.occurred_at < end,
)
qualifying_rows = db.execute(
select(
BehaviorEvent.subject_id,
func.max(BehaviorEvent.occurred_at),
)
.where(*filters)
.group_by(BehaviorEvent.subject_id)
.having(func.count(BehaviorEvent.id) >= threshold)
).all()
qualifying = {str(subject_id) for subject_id, _ in qualifying_rows}
for subject_id, last_at in qualifying_rows:
evaluate_behavior_rule(
db,
rule_code=rule_code,
subject_id=str(subject_id),
at=last_at or current,
)
open_incidents = db.scalars(
select(RiskIncident).where(
RiskIncident.rule_code == rule_code,
RiskIncident.window_key == window_key,
RiskIncident.status == "open",
)
).all()
for incident in open_incidents:
if incident.subject_id not in qualifying:
incident.status = "resolved"
incident.action_reason = AUTO_RESOLVED_REASON
incident.handled_by = None
incident.handled_at = utcnow()
if commit:
db.commit()
return len(qualifying)
def record_behavior_event(
db: Session,
*,
event_type: str,
subject_type: str,
subject_id: str,
user_id: int | None = None,
device_id: str | None = None,
device_model: str | None = None,
phone: str | None = None,
client_ip: str | None = None,
outcome: str = "success",
reason: str | None = None,
details: dict | None = None,
occurred_at: datetime | None = None,
evaluate_rule: str | None = None,
commit: bool = True,
) -> BehaviorEvent:
at = occurred_at or utcnow()
event = BehaviorEvent(
event_type=event_type,
subject_type=subject_type,
subject_id=subject_id,
user_id=user_id,
device_id=device_id,
device_model=device_model,
phone=phone,
client_ip=client_ip,
outcome=outcome,
reason=reason,
details=details,
occurred_at=at,
)
db.add(event)
db.flush()
if evaluate_rule:
evaluate_behavior_rule(db, rule_code=evaluate_rule, subject_id=subject_id, at=at)
if commit:
db.commit()
db.refresh(event)
return event
def sync_compare_incident(
db: Session,
*,
user_id: int,
at: datetime,
threshold: int | None = None,
commit: bool = True,
) -> RiskIncident | None:
effective_threshold = threshold or get_rule_threshold(db, RULE_COMPARE_DAILY)
# comparison_record 的既有写入口统一落“北京时间 naive”时间;这里必须沿用同一
# 口径,否则 SQLite/PG session timezone 不同时会把凌晨记录算到前一天。
local = at.astimezone(CN_TZ).replace(tzinfo=None) if at.tzinfo else at
window_start = local.replace(hour=0, minute=0, second=0, microsecond=0)
end = window_start + timedelta(days=1)
window_key = window_start.strftime("%Y-%m-%d")
reset_at = get_rule_reset_at(db, RULE_COMPARE_DAILY)
reset_local = (
reset_at.astimezone(CN_TZ).replace(tzinfo=None) if reset_at else None
)
start = max(window_start, reset_local) if reset_local else window_start
filters = (
ComparisonRecord.user_id == user_id,
ComparisonRecord.created_at >= start,
ComparisonRecord.created_at < end,
)
count, first_at, last_at = db.execute(
select(
func.count(ComparisonRecord.id),
func.min(ComparisonRecord.created_at),
func.max(ComparisonRecord.created_at),
).where(*filters)
).one()
if int(count or 0) < effective_threshold or first_at is None or last_at is None:
return None
triggered_at = db.execute(
select(ComparisonRecord.created_at)
.where(*filters)
.order_by(ComparisonRecord.created_at.asc(), ComparisonRecord.id.asc())
.offset(effective_threshold - 1)
.limit(1)
).scalar_one()
incident = _upsert_incident(
db,
rule_code=RULE_COMPARE_DAILY,
event_type="compare_start",
subject_type="user",
subject_id=str(user_id),
window_key=window_key,
window_start=start,
window_end=end,
first_event_at=first_at,
triggered_at=triggered_at,
last_event_at=last_at,
event_count=int(count),
)
if commit:
db.commit()
db.refresh(incident)
return incident
def reconcile_compare_rule(
db: Session, *, at: datetime | None = None, commit: bool = True
) -> int:
"""按当前阈值重算北京时间当日比价告警,并收起不再命中的待处理告警。"""
current = (at or utcnow()).astimezone(CN_TZ).replace(tzinfo=None)
window_start = current.replace(hour=0, minute=0, second=0, microsecond=0)
end = window_start + timedelta(days=1)
window_key = window_start.strftime("%Y-%m-%d")
reset_at = get_rule_reset_at(db, RULE_COMPARE_DAILY)
reset_local = (
reset_at.astimezone(CN_TZ).replace(tzinfo=None) if reset_at else None
)
start = max(window_start, reset_local) if reset_local else window_start
threshold = get_rule_threshold(db, RULE_COMPARE_DAILY)
rows = db.execute(
select(
ComparisonRecord.user_id,
func.max(ComparisonRecord.created_at),
)
.where(
ComparisonRecord.user_id.is_not(None),
ComparisonRecord.created_at >= start,
ComparisonRecord.created_at < end,
)
.group_by(ComparisonRecord.user_id)
.having(func.count(ComparisonRecord.id) >= threshold)
).all()
qualifying = {str(user_id) for user_id, _ in rows}
for user_id, last_at in rows:
sync_compare_incident(
db,
user_id=int(user_id),
at=last_at or current,
threshold=threshold,
commit=False,
)
open_incidents = db.scalars(
select(RiskIncident).where(
RiskIncident.rule_code == RULE_COMPARE_DAILY,
RiskIncident.window_key == window_key,
RiskIncident.status == "open",
)
).all()
for incident in open_incidents:
if incident.subject_id not in qualifying:
incident.status = "resolved"
incident.action_reason = AUTO_RESOLVED_REASON
incident.handled_by = None
incident.handled_at = utcnow()
if commit:
db.commit()
return len(qualifying)
def get_active_restriction(
db: Session, *, subject_type: str, subject_id: str, scope: str
) -> SubjectRestriction | None:
return db.execute(
select(SubjectRestriction).where(
SubjectRestriction.subject_type == subject_type,
SubjectRestriction.subject_id == subject_id,
SubjectRestriction.scope.in_((scope, "all")),
SubjectRestriction.active.is_(True),
)
).scalar_one_or_none()
def is_restricted(db: Session, *, subject_type: str, subject_id: str, scope: str) -> bool:
return get_active_restriction(
db, subject_type=subject_type, subject_id=subject_id, scope=scope
) is not None
def activate_restriction(
db: Session,
*,
subject_type: str,
subject_id: str,
scope: str,
reason: str,
incident_id: int | None,
admin_id: int,
) -> SubjectRestriction:
restriction = db.execute(
select(SubjectRestriction).where(
SubjectRestriction.subject_type == subject_type,
SubjectRestriction.subject_id == subject_id,
SubjectRestriction.scope == scope,
)
).scalar_one_or_none()
if restriction is None:
restriction = SubjectRestriction(
subject_type=subject_type,
subject_id=subject_id,
scope=scope,
)
db.add(restriction)
restriction.active = True
restriction.reason = reason
restriction.incident_id = incident_id
restriction.created_by = admin_id
restriction.created_at = utcnow()
restriction.revoked_by = None
restriction.revoked_at = None
db.flush()
return restriction
def revoke_restriction(
db: Session, *, restriction: SubjectRestriction, admin_id: int
) -> None:
restriction.active = False
restriction.revoked_by = admin_id
restriction.revoked_at = utcnow()
db.flush()
+5
View File
@@ -93,6 +93,11 @@ def get_user_by_id(db: Session, user_id: int) -> User | None:
return db.get(User, user_id)
def user_exists(db: Session, user_id: int) -> bool:
"""只查主键判断用户是否存在,避免只读统计接口依赖完整用户表结构。"""
return db.scalar(select(User.id).where(User.id == user_id)) is not None
def get_user_by_phone(db: Session, phone: str) -> User | None:
stmt = select(User).where(User.phone == phone)
return db.execute(stmt).scalar_one_or_none()
+137 -31
View File
@@ -6,6 +6,7 @@
"""
from __future__ import annotations
import logging
import re
import unicodedata
import uuid
@@ -20,7 +21,6 @@ from app.core.config import settings
from app.core.rewards import COIN_PER_CENT, coins_to_cents
from app.integrations import wxpay
from app.models.user import User
from app.repositories.user import apply_wechat_display_identity
from app.models.wallet import (
CashTransaction,
CoinAccount,
@@ -29,13 +29,55 @@ from app.models.wallet import (
WechatTransferAuthorization,
WithdrawOrder,
)
from app.repositories.user import apply_wechat_display_identity
from app.services import notification_events
logger = logging.getLogger(__name__)
# 微信转账终态:成功 / 失败(失败/取消/关闭都退款)
_WX_STATE_SUCCESS = "SUCCESS"
_WX_STATE_FAILED = {"FAIL", "CANCELLED", "CLOSED"}
_WX_STATE_WAIT_CONFIRM = "WAIT_USER_CONFIRM" # 用户还没在微信确认页确认
_WITHDRAW_ACTIVE_STATUSES = {"reviewing", "pending"}
_WX_FAIL_REASON_LABELS = {
"ACCOUNT_FROZEN": "用户微信账户被冻结",
"ACCOUNT_NOT_EXIST": "用户微信账户不存在",
"BANK_CARD_ACCOUNT_ABNORMAL": "用户银行卡已销户、冻结、作废或挂失",
"BANK_CARD_BANK_INFO_WRONG": "用户登记的银行或分支行信息有误",
"BANK_CARD_CARD_INFO_WRONG": "用户银行卡户名或卡号有误",
"BANK_CARD_COLLECTIONS_ABOVE_QUOTA": "用户银行卡收款达到限额",
"BANK_CARD_PARAM_ERROR": "用户收款银行卡信息错误",
"BANK_CARD_STATUS_ABNORMAL": "用户银行卡状态异常",
"BLOCK_B2C_USERLIMITAMOUNT_BSRULE_MONTH": "用户本月转账收款已达限额",
"BLOCK_B2C_USERLIMITAMOUNT_MONTH": "用户账户存在风险,本月收款受限",
"DAY_RECEIVED_COUNT_EXCEED": "用户当日收款次数已达上限",
"DAY_RECEIVED_QUOTA_EXCEED": "用户当日收款额度已达上限",
"EXCEEDED_ESTIMATED_AMOUNT": "转账金额超过预约金额范围",
"ID_CARD_NOT_CORRECT": "收款人身份证校验不通过",
"MCH_CANCEL": "商户已撤销付款",
"MERCHANT_REJECT": "商户转账验密人已驳回",
"MERCHANT_NOT_CONFIRM": "商户转账验密人超时未确认",
"NAME_NOT_CORRECT": "收款人姓名校验不通过",
"OPENID_INVALID": "用户 OpenID 无效或不属于当前 AppID",
"OTHER_FAIL_REASON_TYPE": "微信返回其他失败原因",
"OVERDUE_CLOSE": "超过微信系统重试期,订单自动关闭",
"PAYEE_ACCOUNT_ABNORMAL": "用户微信账户收款异常",
"PAYER_ACCOUNT_ABNORMAL": "商户账户付款受限",
"PRODUCT_AUTH_CHECK_FAIL": "商户未开通转账权限或权限已冻结",
"REALNAME_ACCOUNT_RECEIVED_QUOTA_EXCEED": "用户微信实名账户收款受限",
"REAL_NAME_CHECK_FAIL": "用户未完成微信实名认证",
"RECEIVE_ACCOUNT_NOT_CONFIGURE": "商户未配置收款用户列表",
"RESERVATION_INFO_NOT_MATCH": "转账信息与预约信息不一致",
"RESERVATION_SCENE_NOT_MATCH": "转账场景与预约场景不一致",
"RESERVATION_STATE_INVALID": "预约转账单状态异常",
"TRANSFER_QUOTA_EXCEED": "用户单笔收款额度已达上限",
"TRANSFER_REMARK_SET_FAIL": "微信转账备注设置失败",
"TRANSFER_RISK": "该笔转账存在风险,已被微信拦截",
"TRANSFER_SCENE_INVALID": "商户未获取当前转账场景",
"TRANSFER_SCENE_UNAVAILABLE": "当前转账场景暂不可用",
"RELATED_ORDER_TRANSFER_AMOUNT_EXCEED": "关联订单累计付款金额超过上限",
"RELATED_ORDER_TRANSFER_COUNT_EXCEED": "关联订单累计付款次数超过上限",
"BUDGET_NOT_ENOUGH": "商户预算资金不足",
}
# 占用新人档「一次性」资格的提现状态:进行中(reviewing/pending)或成功打款(success)。
# 被拒/转账失败/解绑退回(rejected/failed,均已退款、钱没到手)不在此列 → 新人档恢复可提
# (2026-07-16 修正:此前判定不看状态,解绑微信退回后 0.1 被误判已用、资格永久锁死)。
@@ -69,10 +111,6 @@ class InsufficientCashError(Exception):
"""现金余额不足。"""
class WithdrawTooFrequentError(Exception):
"""提现申请过于频繁,或已有未完成提现单。"""
class WithdrawTierUnavailableError(Exception):
"""该档位今日不可提:次数已满,或今天已选了其他额度(7-9 福利页档位规则)。"""
@@ -484,6 +522,19 @@ def _try_deduct_cash(db: Session, user_id: int, amount_cents: int, source: str =
def _add_cash(db: Session, user_id: int, amount_cents: int, source: str = "coin_cash") -> int:
"""原子增加指定账户余额(退款用),返回加后余额。source 决定退回哪个账户(两账户隔离)。"""
# Normal withdrawals always create the account before deducting funds. This
# fallback covers legacy rows, hand-written fixtures, and broken migrations:
# a missing balance snapshot must not make a legitimate refund fail with 500.
if db.get(CoinAccount, user_id) is None:
logger.error(
"withdraw refund found missing coin_account; recreating empty account: "
"user_id=%s source=%s amount_cents=%s",
user_id,
source,
amount_cents,
)
get_or_create_account(db, user_id, commit=False)
col = _balance_col(source)
db.execute(
update(CoinAccount)
@@ -587,6 +638,32 @@ def _wx_not_found(result: dict) -> bool:
return "NOT_FOUND" in str(code)
def _wechat_api_error_reason(data: object) -> str:
"""Format a non-200 WeChat API response for operator display."""
if not isinstance(data, dict):
return f"微信发起转账失败:{data}"
code = str(data.get("code") or "").strip()
message = str(data.get("message") or "").strip()
if code and message:
return f"微信发起转账失败:{message}{code}"
return f"微信发起转账失败:{message or code or '未知错误'}"
def _wechat_terminal_failure_reason(data: dict, state: str) -> str:
"""Translate WeChat query ``fail_reason`` while preserving unknown codes."""
code = str(data.get("fail_reason") or "").strip()
if code:
label = _WX_FAIL_REASON_LABELS.get(code)
if label:
return f"微信转账失败:{label}{code}"
return f"微信转账失败:{code}"
if state == "CANCELLED":
return "微信转账已撤销(CANCELLED"
if state == "CLOSED":
return "微信转账已关闭(CLOSED"
return f"微信转账失败(状态:{state or 'FAIL'}"
def _settle_after_ambiguous(db: Session, order: WithdrawOrder, reason: str) -> None:
"""转账调用结果不明(超时/异常/非200)时,**先查单再决定**,绝不盲目退款(防退款后又到账)。
- 微信查到 SUCCESS 钱已出, success,不退款
@@ -612,13 +689,17 @@ def _settle_after_ambiguous(db: Session, order: WithdrawOrder, reason: str) -> N
state = q["data"].get("state", "")
order.wechat_state = state
order.transfer_bill_no = q["data"].get("transfer_bill_no") or order.transfer_bill_no
if state == _WX_STATE_SUCCESS:
order.status = "success"
order.transfer_bill_no = q["data"].get("transfer_bill_no")
db.commit()
notification_events.notify_withdraw_success(db, order) # PRD #3 提现到账
elif state in _WX_STATE_FAILED:
_refund_withdraw(db, order, reason=reason)
_refund_withdraw(
db,
order,
reason=_wechat_terminal_failure_reason(q["data"], state),
)
else:
order.package_info = q["data"].get("package_info") or order.package_info
db.commit()
@@ -755,17 +836,8 @@ def create_withdraw(
else:
out_bill_no = uuid.uuid4().hex
active_order_id = db.execute(
select(WithdrawOrder.id).where(
WithdrawOrder.user_id == user_id,
WithdrawOrder.status.in_(_WITHDRAW_ACTIVE_STATUSES),
).limit(1)
).scalar_one_or_none()
if active_order_id is not None:
raise WithdrawTooFrequentError
# 福利页档位闸(7-9):coin_cash 只能提预设档位,且该档今日可提(服务端权威口径,防绕过
# 客户端刷)。放在幂等返回/在途互斥之后:同号重试仍原样返回旧单,不被档位闸误杀。
# 客户端刷)。放在幂等返回之后:同号重试仍原样返回旧单,不被档位闸误杀。
# allow_sub_min(0.01 调试直发)保持原样放行,不受档位约束;invite_cash 本轮无档位概念不校验。
if source == "coin_cash" and not allow_sub_min:
tier_state = next(
@@ -815,6 +887,7 @@ def create_withdraw(
db.commit()
except IntegrityError:
db.rollback()
# 唯一冲突只可能来自 out_bill_no 幂等键并发重试:原样返回既有单;否则未知冲突,上抛。
existing = db.execute(
select(WithdrawOrder).where(
WithdrawOrder.out_bill_no == out_bill_no, WithdrawOrder.user_id == user_id
@@ -822,14 +895,6 @@ def create_withdraw(
).scalar_one_or_none()
if existing is not None:
return existing
active_order_id = db.execute(
select(WithdrawOrder.id).where(
WithdrawOrder.user_id == user_id,
WithdrawOrder.status.in_(_WITHDRAW_ACTIVE_STATUSES),
).limit(1)
).scalar_one_or_none()
if active_order_id is not None:
raise WithdrawTooFrequentError from None
raise
db.refresh(order)
return order # 待管理员审核;**不在此处打款**
@@ -988,6 +1053,14 @@ def _apply_transfer_result(db: Session, order: WithdrawOrder, data: dict) -> Wit
order.package_info = data.get("package_info") # 免确认转账无此字段(None);确认模式带它供拉确认页
if data.get("state") == _WX_STATE_SUCCESS:
order.status = "success"
elif data.get("state") in _WX_STATE_FAILED:
_refund_withdraw(
db,
order,
reason=_wechat_terminal_failure_reason(data, str(data.get("state") or "")),
)
db.refresh(order)
return order
db.commit()
db.refresh(order)
if order.status == "success": # 免确认转账直接到账 → PRD #3 提现到账
@@ -1032,7 +1105,11 @@ def execute_withdraw_transfer(db: Session, order: WithdrawOrder) -> WithdrawOrde
return order
if result["status_code"] != 200:
# 金额安全:查转账单后定夺,绝不盲退(未创建→退款,已创建→按真实状态)
_settle_after_ambiguous(db, order, reason=str(result["data"].get("message") or result["data"]))
_settle_after_ambiguous(
db,
order,
reason=_wechat_api_error_reason(result["data"]),
)
# 授权有效性:回查授权单,微信侧已失效(用户关闭/风控)→标 closed,下次提现自动回退方式一重新授权
_refresh_active_auth(db, order.user_id)
db.refresh(order)
@@ -1060,7 +1137,11 @@ def execute_withdraw_transfer(db: Session, order: WithdrawOrder) -> WithdrawOrde
return order
if result["status_code"] != 200:
_settle_after_ambiguous(db, order, reason=str(result["data"].get("message") or result["data"]))
_settle_after_ambiguous(
db,
order,
reason=_wechat_api_error_reason(result["data"]),
)
db.refresh(order)
return order
@@ -1117,19 +1198,24 @@ def refresh_withdraw_status(
).scalar_one_or_none()
if order is None:
raise WithdrawOrderNotFound
if order.status != "pending":
return order # 已终态,不再查
if order.status not in {"pending", "failed"}:
return order
enrich_failed_order = order.status == "failed"
try:
result = wxpay.query_transfer(out_bill_no)
except wxpay.WxPayNotConfiguredError:
raise
except Exception as exc: # noqa: BLE001 - 查单失败不能把运营后台打成 500
if enrich_failed_order:
return order
order.fail_reason = f"微信查单异常,保持pending: {exc}"[:256]
db.commit()
db.refresh(order)
return order
if result["status_code"] != 200:
if enrich_failed_order:
return order
if _wx_not_found(result):
# 微信明确无此单 → 转账从未创建(如崩溃在扣款后/调用前),退款安全
_refund_withdraw(db, order, reason="微信无此单,已退回")
@@ -1138,12 +1224,32 @@ def refresh_withdraw_status(
state = result["data"].get("state", "")
order.wechat_state = state
order.transfer_bill_no = (
result["data"].get("transfer_bill_no") or order.transfer_bill_no
)
if enrich_failed_order:
if state in _WX_STATE_FAILED:
order.fail_reason = _wechat_terminal_failure_reason(
result["data"], state
)[:256]
elif state == _WX_STATE_SUCCESS:
order.fail_reason = (
"资金状态异常:本地已退款,但微信查单显示已到账,请人工核查"
)
db.commit()
db.refresh(order)
return order
if state == _WX_STATE_SUCCESS:
order.status = "success"
db.commit()
notification_events.notify_withdraw_success(db, order) # PRD #3 提现到账
elif state in _WX_STATE_FAILED:
_refund_withdraw(db, order, reason=f"微信转账状态 {state}")
_refund_withdraw(
db,
order,
reason=_wechat_terminal_failure_reason(result["data"], state),
)
elif state == _WX_STATE_WAIT_CONFIRM and cancel_if_unconfirmed:
# 用户从确认页回来了却仍未确认 → 视为放弃:撤销微信单(防事后确认导致重复打款)后退款。
# 撤单失败(可能已被确认进 ACCEPTED 的竞态)则保持 pending,等下次查询。
+31
View File
@@ -0,0 +1,31 @@
"""客户端运行日志批量上报 schema。
批级公共字段(device_id/user_id/app_ver/platform/sent_at)发一次;logs 为原始 dict 列表,
**不强类型**尽力而为的日志链路,单条内容异常不该让整批 422每条的白名单键 + data
兜底拆分在写入层 [app.core.client_log] ( spec §4/§5)
"""
from __future__ import annotations
import os
from typing import Any
from pydantic import BaseModel, Field
# 单批条数上限。注意:这是**导入期**常量(Pydantic Field(max_length=) 在类定义时求值),
# 改它需重启进程;要运行期可调的上限用 APPLOG_MAX_BODY_BYTES(端点里 call-time 读)。
_MAX_BATCH = int(os.getenv("APPLOG_MAX_BATCH", "500"))
class AppLogBatchIn(BaseModel):
device_id: str = Field(max_length=64)
user_id: int | None = None
app_ver: str | None = Field(default=None, max_length=32)
platform: str | None = Field(default=None, max_length=16)
sent_at: int | None = None
logs: list[dict[str, Any]] = Field(min_length=1, max_length=_MAX_BATCH)
class AppLogIngestOut(BaseModel):
ok: bool = True
received: int
dropped: int = 0
+9 -1
View File
@@ -11,7 +11,6 @@ from datetime import datetime
from pydantic import BaseModel, ConfigDict, Field
# ===== 用户对外信息 =====
class UserOut(BaseModel):
@@ -65,6 +64,9 @@ class JverifyLoginRequest(BaseModel):
"", max_length=64,
description="硬件级设备标识(Android ANDROID_ID),用于新手引导按 设备+账号 去重;空=按未完成处理",
)
device_model: str = Field(
"", max_length=128, description="客户端设备型号快照,用于登录安全审计"
)
# ===== 短信验证码 =====
@@ -75,6 +77,9 @@ class SmsSendRequest(BaseModel):
"", max_length=64,
description="硬件级设备标识(Android ANDROID_ID),用于发码防刷按 设备+IP 限流;空=按 IP 聚一桶",
)
device_model: str = Field(
"", max_length=128, description="客户端设备型号快照,用于短信安全审计"
)
class SmsSendResponse(BaseModel):
@@ -90,6 +95,9 @@ class SmsLoginRequest(BaseModel):
"", max_length=64,
description="硬件级设备标识(Android ANDROID_ID),用于新手引导按 设备+账号 去重;空=按未完成处理",
)
device_model: str = Field(
"", max_length=128, description="客户端设备型号快照,用于短信验证安全审计"
)
# ===== Refresh =====
+30 -1
View File
@@ -13,7 +13,6 @@ from datetime import datetime
from pydantic import BaseModel, ConfigDict, Field, field_validator
# ===== 上报请求 =====
class ComparisonItemIn(BaseModel):
@@ -86,6 +85,10 @@ class ComparisonResultIn(BaseModel):
status: str | None = None
# 门店打烊原因(price 为 None 时带): 与 status="store_closed" 等价的更早信号, 一并落库供前端兜底判打烊。
store_closed: str | None = None
# 该平台缺菜(漏菜)数量(pricebot 冗余进 comparison_results 行): 有价但少买了菜时 >0。
# 记录页三平台网格据此逐格标"缺少 X 个菜品"(网格只拿 comparison_results, 拿不到 platform_results)。
# 必须显式声明: 落库走 model_dump(), 不声明会被 pydantic 静默丢弃 → 记录页拿不到数量。
skipped_dish_count: int | None = None
class ComparisonRecordIn(BaseModel):
@@ -104,6 +107,13 @@ class ComparisonRecordIn(BaseModel):
# 明细
items: list[ComparisonItemIn] = Field(default_factory=list)
comparison_results: list[ComparisonResultIn] = Field(default_factory=list)
# 展示模型统一数组(pricebot done.params.platforms 原样透传): 每平台一行、自带
# status/is_best/display/display_order,记录页据此直渲染。宽松 list[dict] 存(结构由
# pricebot 定,server 只原样落库),前端读它、老记录空时回退 comparison_results。
platforms: list[dict] = Field(default_factory=list)
# 记录级结局(pricebot 下发): success/below_minimum/store_closed/failed。让"未满起送"不再
# 被塌缩成 failed。_derive 优先用它、其次客户端 status、再兜底二态派生。
record_status: str | None = None
# 逐平台结局摘要(含失败平台的细分原因 status: store_not_found/items_not_found/below_minimum/
# unsupported/...)。来自 done.params.platform_results,客户端透传;落 raw_payload(不单列),
# admin「卡在哪一步」从这里读。dict{platform_id: {...}} 宽松存(结构由 pricebot 定——是
@@ -170,8 +180,13 @@ class ComparisonRecordOut(BaseModel):
skipped_dish_count: int | None = None
status: str
information: str | None = None
# 失败卡「原因」文案:具体失败给具体原因,纯系统失败为 None(端侧品牌兜底)。见模型 fail_reason。
fail_reason: str | None = None
items: list = []
comparison_results: list = []
# 展示模型统一数组(每平台一行、自带 status/is_best/display/display_order): 记录页据此
# 直渲染, 不再靠 comparison_results + 前端派生。老记录为空 → 前端回退 comparison_results。
platforms: list = []
skipped_dish_names: list = []
total_ms: int | None = None
# 「已下单」(店级):该店名在该用户真实下单(source='compare')里出现过即 True。
@@ -198,6 +213,20 @@ class ComparisonRecordCreatedOut(BaseModel):
id: int = Field(..., description="写入(或已存在)的记录 id")
class CompareStartReserveIn(BaseModel):
"""Reserve one authenticated comparison start before the agent begins."""
trace_id: str = Field(..., min_length=1, max_length=64)
business_type: str = Field(default="food", min_length=1, max_length=16)
device_id: str | None = Field(default=None, max_length=64)
class CompareStartReserveOut(BaseModel):
limit: int
used: int
remaining: int
class CompareStatsOut(BaseModel):
"""「我的」页省钱战绩卡(比价口径)聚合。"""
+36
View File
@@ -0,0 +1,36 @@
"""新手引导视频(领券等候浮层前 N 次替代广告)的客户端请求/响应契约。"""
from __future__ import annotations
from pydantic import BaseModel, Field
class GuideVideoStartIn(BaseModel):
"""开播询问。scene 目前只有 coupon(领券浮层);预留给日后比价等场景。"""
scene: str = Field(default="coupon", max_length=16)
class GuideVideoStartOut(BaseModel):
"""should_play=False 时客户端照旧走广告链路,其余字段无意义。"""
should_play: bool
video_url: str | None = None # 相对地址 /media/...;客户端自行拼 BASE_URL
play_token: str = "" # 发奖幂等键
reward_coin: int = 0 # 播完/中途关闭都发的固定金币
seq: int = 0 # 本账号第几次
remaining: int = 0 # 发完这次还剩几次
class GuideVideoRewardIn(BaseModel):
"""播完或中途关闭都调这个;completed 只做留痕,两者都发币。"""
play_token: str = Field(min_length=1, max_length=64)
completed: bool = False
class GuideVideoRewardOut(BaseModel):
"""granted=True 表示本次调用真的入账(重复上报为 False,coin 是已发金额)。"""
granted: bool
coin: int
status: str
+156
View File
@@ -0,0 +1,156 @@
"""Persist and repair comparison-record LLM token costs."""
from __future__ import annotations
import logging
import time
from datetime import UTC, datetime, timedelta
from sqlalchemy import select
from app.core.config import settings
from app.core.rewards import CN_TZ
from app.db.session import SessionLocal
from app.models.app_config import AppConfig
from app.models.comparison import ComparisonRecord
from app.services.llm_cost import compute_llm_cost, get_llm_prices
from app.services.pricebot_llm_calls import fetch_llm_calls
logger = logging.getLogger("shagua.comparison_llm_backfill")
def _utc_to_beijing_naive(value: datetime) -> datetime:
"""Convert a DB UTC timestamp to comparison_record's Beijing wall-clock."""
if value.tzinfo is None:
value = value.replace(tzinfo=UTC)
return value.astimezone(CN_TZ).replace(tzinfo=None)
def _store_calls(record_id: int, trace_id: str, calls: list[dict]) -> bool:
"""Store calls and all derived fields atomically."""
with SessionLocal() as db:
rec = db.get(ComparisonRecord, record_id)
if rec is None or rec.trace_id != trace_id:
logger.warning(
"LLM cost backfill record mismatch record_id=%s trace=%s",
record_id,
trace_id,
)
return False
# Never recalculate a frozen historical cost with a newer price config.
if rec.llm_cost_yuan is not None and rec.llm_calls:
return False
rec.llm_calls = calls
rec.llm_call_count = len(calls)
rec.retry_count = sum(1 for call in calls if call.get("error"))
rec.input_tokens = sum(
(call.get("usage") or {}).get("prompt_tokens") or 0 for call in calls
)
rec.output_tokens = sum(
(call.get("usage") or {}).get("completion_tokens") or 0 for call in calls
)
rec.llm_cost_yuan, rec.llm_price_snapshot = compute_llm_cost(
calls, get_llm_prices(db)
)
db.commit()
logger.info(
"LLM cost backfilled trace=%s calls=%d input_tokens=%d "
"output_tokens=%d cost=%s",
trace_id,
len(calls),
rec.input_tokens,
rec.output_tokens,
rec.llm_cost_yuan,
)
return True
def backfill_comparison_llm_cost(
record_id: int,
trace_id: str,
*,
attempts: int = 3,
retry_delays: tuple[float, ...] = (1.0, 3.0),
) -> bool:
"""Fetch and persist one record, retrying short-lived upstream races."""
if not settings.INTERNAL_API_SECRET or not trace_id:
logger.warning(
"LLM cost backfill skipped trace=%s: INTERNAL_API_SECRET is not configured",
trace_id,
)
return False
total_attempts = max(1, attempts)
for attempt in range(total_attempts):
calls = fetch_llm_calls(trace_id)
if calls:
try:
return _store_calls(record_id, trace_id, calls)
except Exception: # noqa: BLE001 - background repair must stay alive
logger.exception(
"LLM cost store failed trace=%s record_id=%s",
trace_id,
record_id,
)
return False
if attempt + 1 < total_attempts:
delay = retry_delays[min(attempt, len(retry_delays) - 1)] if retry_delays else 0
if delay > 0:
time.sleep(delay)
logger.warning(
"LLM cost backfill has no calls trace=%s record_id=%s attempts=%d",
trace_id,
record_id,
total_attempts,
)
return False
def repair_missing_comparison_llm_costs(
*,
limit: int = 100,
lookback_days: int = 30,
) -> dict[str, int]:
"""Repair a bounded batch of recent terminal records with missing cost."""
cutoff = datetime.now(CN_TZ).replace(tzinfo=None) - timedelta(
days=max(1, lookback_days)
)
with SessionLocal() as db:
# app_config has no price history. Repricing a record from before the
# current config became effective would fabricate a historical cost, so
# only repair records at/after that timestamp.
price_config_updated_at = db.execute(
select(AppConfig.updated_at).where(AppConfig.key == "llm_token_price")
).scalar_one_or_none()
date_conditions = [ComparisonRecord.created_at >= cutoff]
if price_config_updated_at is not None:
date_conditions.append(
ComparisonRecord.created_at
>= _utc_to_beijing_naive(price_config_updated_at)
)
candidates = list(
db.execute(
select(ComparisonRecord.id, ComparisonRecord.trace_id)
.where(
*date_conditions,
ComparisonRecord.status.in_(("success", "failed", "cancelled")),
ComparisonRecord.llm_cost_yuan.is_(None),
)
.order_by(ComparisonRecord.created_at.desc(), ComparisonRecord.id.desc())
.limit(max(1, limit))
).all()
)
repaired = 0
for record_id, trace_id in candidates:
if backfill_comparison_llm_cost(
record_id, trace_id, attempts=1, retry_delays=()
):
repaired += 1
return {
"candidates": len(candidates),
"repaired": repaired,
"unresolved": len(candidates) - repaired,
}
+57 -9
View File
@@ -20,19 +20,67 @@ from app.core.pricebot_router import pick_pricebot
logger = logging.getLogger("shagua.pricebot_llm")
def pricebot_llm_auth_ready() -> bool:
"""Verify every configured PriceBot instance accepts the shared secret."""
secret = settings.INTERNAL_API_SECRET
if not secret:
logger.error("PriceBot LLM auth check failed: INTERNAL_API_SECRET is empty")
return False
for base in settings.pricebot_instances:
url = f"{base.rstrip('/')}/api/internal/llm_calls/__auth_probe__"
try:
resp = httpx.get(
url, headers={"X-Internal-Secret": secret}, timeout=3.0
)
except Exception as exc: # noqa: BLE001
logger.error("PriceBot LLM auth check unavailable base=%s: %s", base, exc)
return False
if resp.status_code != 200:
logger.error(
"PriceBot LLM auth check rejected base=%s status=%s; "
"verify both services use the same INTERNAL_API_SECRET",
base,
resp.status_code,
)
return False
return True
def fetch_llm_calls(trace_id: str) -> list[dict]:
"""返回该次比价的 LLM 调用明细列表(每条 {scene,model,input_messages,output,usage,latency_ms,error});
未配密钥 / trace_id / 拉取失败 []"""
secret = settings.INTERNAL_API_SECRET
if not secret or not trace_id:
return []
base = pick_pricebot(trace_id).rstrip("/")
url = f"{base}/api/internal/llm_calls/{trace_id}"
try:
resp = httpx.get(url, headers={"X-Internal-Secret": secret}, timeout=5.0)
if resp.status_code == 200:
return resp.json().get("calls", []) or []
logger.warning("fetch_llm_calls trace=%s status=%s", trace_id, resp.status_code)
except Exception as e: # noqa: BLE001 — best-effort,任何异常都不该影响上报
logger.warning("fetch_llm_calls trace=%s failed: %s", trace_id, e)
preferred = pick_pricebot(trace_id)
# LLM JSONL is instance-local. If the cluster topology changed after a
# historical trace was created, consistent hashing may now point elsewhere;
# probe the remaining configured instances only when the preferred one is empty.
bases = [preferred, *(base for base in settings.pricebot_instances if base != preferred)]
for base in bases:
url = f"{base.rstrip('/')}/api/internal/llm_calls/{trace_id}"
try:
resp = httpx.get(url, headers={"X-Internal-Secret": secret}, timeout=5.0)
if resp.status_code == 200:
calls = resp.json().get("calls", []) or []
if calls:
return calls
continue
if resp.status_code in (401, 403):
logger.error(
"fetch_llm_calls rejected trace=%s base=%s status=%s; "
"INTERNAL_API_SECRET differs between app-server and PriceBot",
trace_id,
base,
resp.status_code,
)
else:
logger.warning(
"fetch_llm_calls trace=%s base=%s status=%s",
trace_id,
base,
resp.status_code,
)
except Exception as e: # noqa: BLE001 — best-effort
logger.warning("fetch_llm_calls trace=%s base=%s failed: %s", trace_id, base, e)
return []
+97
View File
@@ -0,0 +1,97 @@
"""美团搜索翻页游标(searchId)缓存 —— 「距离最近」tab 翻页提速。
**问题**:美团 query_coupon 的搜索结果只能靠 searchId 续页(pageNo 翻不动),而我们的 HTTP 接口
是无状态的客户端只传页码原实现因此每次都从第 1 页顺序重放到第 N 取第 N 页要向美团发
N 次请求,用户越往下滑越慢( 5 1 页的 5 倍耗时,且每次调用都可能撞 402 限流)
**做法**:翻到第 n 页所需的 searchId (量化坐标, platform, 关键词) 记下来,
下次请求第 n 页直接一发命中;未命中才从缓存里最深的那一页往后重放,并把沿途游标补进缓存
稳态下每翻一页恒定 1 次请求
**坐标量化** 2 位小数(~1km), `utils/meituan_city` 同口径:原始 GPS 每次抖动到小数点后 5~6 ,
不量化几乎不命中缓存;而同一路搜索的排序原点相差 1km 以内,对券列表的实际顺序无感知差别
**TTL** 10 分钟:searchId 属于上游会话态,放太久会翻到过期游标(调用失败 调用方回退重放)
缓存是**进程内**, worker 各存一份不共享,这没问题:命中率只影响快慢,不影响正确性
"""
from __future__ import annotations
import threading
import time
# 缓存键:(量化纬度, 量化经度, platform, 搜索词)
RouteKey = tuple[float, float, int, str]
_TTL_SEC = 10 * 60
_MAX_ROUTES = 512 # 最多缓存多少条搜索路线(每条 = 一个坐标×平台×关键词)
_MAX_PAGES_PER_ROUTE = 60 # 单条路线最多记多少页游标,防某个坐标被无限翻页撑爆
# {route_key: {page_no: (search_id, 写入时刻)}};page_no 表示「用这个 searchId 能取到第几页」。
# 第 1 页不需要 searchId(直接 pageNo=1),故永远不入表。
_cursors: dict[RouteKey, dict[int, tuple[str, float]]] = {}
_lock = threading.Lock()
def route_key(latitude: float, longitude: float, platform: int, keyword: str) -> RouteKey:
"""构造缓存键(坐标量化到 ~1km)。"""
return (round(latitude, 2), round(longitude, 2), platform, keyword)
def lookup(key: RouteKey, page: int) -> tuple[int, str | None]:
"""找到能最省调用地拿到第 `page` 页的起点。
返回 (起始页, 该页要用的 search_id):
- (page, "xxx") 缓存命中 调用方一发直达第 page
- (m, "xxx") 命中更浅的第 m (1 < m < page) 从第 m 页重放到第 page
- (1, None) 全未命中 从第 1 (pageNo=1)重放
"""
if page <= 1:
return 1, None
now = time.time()
with _lock:
pages = _cursors.get(key)
if not pages:
return 1, None
for p in range(page, 1, -1):
hit = pages.get(p)
if hit is None:
continue
search_id, wrote_at = hit
if now - wrote_at > _TTL_SEC:
pages.pop(p, None) # 过期即清,继续往浅了找
continue
return p, search_id
return 1, None
def remember(key: RouteKey, page: int, search_id: str) -> None:
"""记下「用 search_id 可取到第 page 页」。page<=1 无意义(第 1 页不用游标)。"""
if page <= 1 or not search_id:
return
now = time.time()
with _lock:
pages = _cursors.get(key)
if pages is None:
if len(_cursors) >= _MAX_ROUTES:
_evict_oldest_route_locked()
pages = _cursors[key] = {}
pages[page] = (search_id, now)
if len(pages) > _MAX_PAGES_PER_ROUTE:
# 越浅的页越容易被重新走到,优先丢最深的那些(它们下次多半也过期了)
for p in sorted(pages, reverse=True)[: len(pages) - _MAX_PAGES_PER_ROUTE]:
pages.pop(p, None)
def drop(key: RouteKey) -> None:
"""整条路线作废。用在「拿缓存游标去请求却失败」时(多半是上游 searchId 过期)。"""
with _lock:
_cursors.pop(key, None)
def _evict_oldest_route_locked() -> None:
"""淘汰最久没更新过的一条路线(调用方须已持锁)。"""
oldest_key = min(
_cursors,
key=lambda k: max((w for _, w in _cursors[k].values()), default=0.0),
)
_cursors.pop(oldest_key, None)
+40
View File
@@ -0,0 +1,40 @@
# 比价 TOKEN 成本采集与补偿
## 部署前置
App Server 与 PriceBot 使用各自独立的 `.env`,但下面的值必须完全一致:
- `/opt/shaguabijia-app-server/.env`
- `/opt/pricebot-backend/.env`
- 配置项:`INTERNAL_API_SECRET`
不要把密钥原文写入日志、命令历史或 Git。修改后同时重启两个服务。
App Server 启动后会逐个探测 `PRICEBOT_INSTANCES` 的内部读取接口。鉴权不一致时会记录
`PriceBot LLM auth check rejected`,并跳过本轮补偿,避免对所有缺失记录重复发送失败请求。
## 数据链路
1. 当前客户端由 App Server 在 PriceBot 最终 `done` 帧到达时 harvest 比价记录。
2. harvest 成功后立即异步读取同一 `trace_id` 的 LLM 调用,冻结 Token、成本和单价快照。
3. 周期 worker 扫描近期 `success/failed``llm_cost_yuan IS NULL` 的记录进行补偿;
为避免用现价伪造历史成本,只处理当前单价配置生效时间之后的记录。
4. 管理后台顶部“平均 TOKEN 成本”使用筛选范围内已冻结成本的数据库平均值。
## 上线验收(只读 SQL
```sql
SELECT
(created_at AT TIME ZONE 'Asia/Shanghai')::date AS day,
count(*) AS records,
count(llm_cost_yuan) AS cost_records,
round(avg(llm_cost_yuan)::numeric, 6) AS avg_token_cost
FROM comparison_record
WHERE created_at >= now() - interval '3 days'
GROUP BY 1
ORDER BY 1 DESC;
```
新产生的正常终态比价记录应在短时间内写入 `input_tokens``output_tokens`
`llm_cost_yuan`。历史记录只有在 PriceBot 的对应 trace JSONL 仍保留时才能准确回填;
原始调用已经清理的记录不能用估算值冒充真实成本。
+4 -23
View File
@@ -1,13 +1,9 @@
# 傻瓜比价 App 后端 — API 接口文档(索引)
> Base URL:生产 `https://app-api.shaguabijia.com`;本地联调 `http://<开发机>:8770`
> 协议:HTTP / JSON,请求与响应体均 `application/json`,字段统一 **snake_case**(⚠️ 例外:消息通知中心 `notifications` 族与厂商推送 `push` 族按 PRD 前端契约用 **camelCase**,见各自文档)
> 协议:HTTP / JSON,请求与响应体均 `application/json`,字段统一 **snake_case**
> 鉴权:需鉴权的接口在请求头带 `Authorization: Bearer <access_token>`
<<<<<<< HEAD
> 最后更新:2026-07-14(新增 **消息通知中心** 3 端点(M1-M3,虚拟数据阶段)与 **厂商推送测试** 3 端点(P1-P3,荣耀/华为/小米/OPPO/vivo);上一次 2026-06-23 补全 device/internal/CPS 短链等整族端点)
=======
> 最后更新:2026-07-09(① 比价透传改「软鉴权 + trace_id 签发 + harvest 落库」(#112 尾声帧 `trace/epilogue` 一并补录);② 新端点:`user/onboarding/reset`(#114)、`GET /internal/launch-confirm-samples`(#91);③ 参数更新:提现族 `source` 分账(#82/#121)、`wallet/account` 邀请奖励金余额、美团 feed/top-sales 按城市过滤(#116)、admin 调现金 `account` 目标账户(#95);④ **Admin 索引补全到当前全量**:新家族 roles(#117/#126)/coupon-data(#99)/device-liveness(#80)/event-logs(#83)/price-reports(#94)/CPS 运营台/提现审核族,及 feedbacks 采纳拒绝(#94/#105)、marquee 模式与真实条浏览(#122/#123)等。上一次 2026-07-03
>>>>>>> origin/main
> 架构:`app/api/v1/` 只放很轻的接口层;穿山甲/微信支付/极光/短信/美团等 SDK 集成的重逻辑在 `app/integrations/`,实现细节见 [docs/integrations/](../integrations/README.md)。
---
@@ -82,6 +78,7 @@
| **签到**(前缀 `/api/v1/signin` |||
| 25 | `GET /api/v1/signin/status` | Bearer | [详情](./signin/signin-status.md) |
| 26 | `POST /api/v1/signin` | Bearer | [详情](./signin/signin-do.md) |
| 26a | `POST /api/v1/signin/boost` | Bearer | [详情](./signin/signin-boost.md) |
| **任务**(前缀 `/api/v1/tasks` |||
| 27 | `GET /api/v1/tasks` | Bearer | [详情](./tasks/tasks-list.md) |
| 28 | `POST /api/v1/tasks/{task_key}/claim` | Bearer | [详情](./tasks/tasks-claim.md) |
@@ -92,7 +89,6 @@
| **看广告发奖**(前缀 `/api/v1/ad` |||
| 32 | `GET /api/v1/ad/pangle-callback` | 验签 | [详情](./ad/ad-pangle-callback.md) |
| 33 | `GET /api/v1/ad/reward-status` | Bearer | [详情](./ad/ad-reward-status.md) |
| 33a | `GET /api/v1/ad/reward-result/{ad_session_id}` | Bearer | [详情](./ad/ad-reward-result.md)(本次实发金币 + 本轮膨胀累计 `round_coin`,弹窗数字用它) |
| 34 | `POST /api/v1/ad/test-grant` | Bearer | [详情](./ad/ad-test-grant.md) |
| 35 | `POST /api/v1/ad/ecpm-report` | Bearer | [详情](./ad/ad-ecpm-report.md) |
| 35a | `POST /api/v1/ad/feed-reward` | Bearer | [详情](./ad/ad-feed-reward.md) |
@@ -107,33 +103,19 @@
| 36c | `POST /api/v1/user/onboarding/reset` | Bearer | [详情](./user/user-onboarding.md)(重置本设备引导标记,下次登录重走,#114 |
| 37 | `DELETE /api/v1/user` | Bearer | [详情](./user/user-delete.md) |
| **帮助与反馈**(前缀 `/api/v1/feedback` |||
<<<<<<< HEAD
| 38 | `POST /api/v1/feedback` | Bearer | [详情](./feedback.md) |
| 38a | `GET /api/v1/feedback/config` | Bearer | 反馈页「加群二维码」卡配置(开关 + 二维码图 + 三行文案)(无单独文档) |
| 38b | `GET /api/v1/feedback/records` | Bearer | 我的反馈历史(pending/adopted/rejected(无单独文档) |
| **消息通知中心**(前缀 `/api/v1/notifications`;⚠️ 本族对外 **camelCase**;虚拟数据阶段:内存 mock,重启复位) |||
| M1 | `GET /api/v1/notifications` | Bearer | [详情](./notifications.md)(消息列表,分页;13 类型卡片字段 + sentAt/isRead;服务端已按时间倒序排好,不分组) |
| M2 | `GET /api/v1/notifications/unread-count` | Bearer | [详情](./notifications.md)(未读总数,首页铃铛角标;>99 → "99+",0 → null 隐藏) |
| M3 | `POST /api/v1/notifications/read` | Bearer | [详情](./notifications.md)(标记已读:`{ids:[...]}` 单条/多条 或 `{all:true}` 进通知中心全量清零;幂等) |
| **厂商推送测试**(前缀 `/api/v1/push`;荣耀/华为/小米/OPPO/vivo 五通道联调三件套,同为 camelCase |||
| P1 | `GET /api/v1/push/vendors` | Bearer | [详情](./push-vendor-test.md)(5 厂商服务端凭据配置状态,缺哪些 .env 键一目了然) |
| P2 | `GET /api/v1/push/templates` | Bearer | [详情](./push-vendor-test.md)(13 类通知的 push 标题/正文模板 + PRD 示例渲染效果) |
| P3 | `POST /api/v1/push/test` | Bearer | [详情](./push-vendor-test.md)(测试发送:默认 mock 不真发;mock=false 真发;可联动插一条站内 mock 通知闭环验证已读) |
=======
| 38 | `POST /api/v1/feedback` | Bearer | [详情](./other/feedback.md) |
| 38a | `GET /api/v1/feedback/config` | Bearer | [详情](./other/feedback-config.md)(反馈页「加群二维码」卡配置:开关+二维码图+三行文案) |
| 38b | `GET /api/v1/feedback/records` | Bearer | [详情](./other/feedback-records.md)(我的反馈历史,pending/adopted/rejected |
| **埋点 & 订单上报**(前缀分散;全部 Bearer 除 analytics/events 不强制登录) |||
| **埋点 & 订单上报 / 客户端日志**(前缀分散;全部 Bearer 除 analytics/events、applog/batch 不强制登录) |||
| E1 | `POST /api/v1/analytics/events` | 无 | [详情](./other/analytics-events.md)(批量上报埋点事件,不强制登录,每批最多200条) |
| E2 | `POST /api/v1/order/report` | Bearer | [详情](./other/order-report.md)(上报归因订单,比价后5分钟内点链接+支付金额与比价价相差≤1元) |
>>>>>>> origin/main
| E3 | `POST /api/v1/applog/batch` | 无 | [详情](./other/applog-batch.md)(批量上报客户端运行日志,逐条落独立文件供 Logtail 采进 SLS,不强制登录,每批≤500条) |
| **首页门面数据 / 客户端配置**(前缀 `/api/v1/platform`;全平台展示数字 + 运营开关,**全部不鉴权**,登录前可读) |||
| 39 | `GET /api/v1/platform/stats` | 无 | [详情](./platform/platform-stats.md) |
| 40 | `GET /api/v1/platform/savings-feed` | 无 | [详情](./savings/platform-savings-feed.md) |
| 40a | `GET /api/v1/platform/flags` | 无 | [详情](./platform/platform-flags.md)(客户端运营 feature flag,比价/领券期广告开关等,拉取后缓存) |
| 40b | `GET /api/v1/platform/ad-config` | 无 | [详情](./platform/platform-ad-config.md)(客户端拉广告配置:穿山甲 app_id+各位ID+各场景开关;不含验签密钥) |
| 40c | `GET /api/v1/platform/app-version` | 无 | [详情](./platform/platform-app-version.md)(最新 App 版本,OTA 检查更新;与本机 versionCode 比) |
| 40d | `GET /api/v1/platform/huawei-review` | 无 | [详情](./platform/platform-huawei-review.md)(华为审核开关:快速设置权限步能否被用户关闭;仅华为 ROM 客户端拉) |
| **微信支付回调**(前缀 `/api/v1/wxpay` |||
| W1 | `POST /api/v1/wxpay/transfer-auth-notify` | 无 | 免确认收款授权结果通知(一期 stub:仅应答 200 不验签不改账,授权状态靠主动查询兜底)(无单独文档) |
| **CPS 群发短链落地**(**无前缀**,挂域名根;公网不鉴权) |||
@@ -172,7 +154,6 @@
| A12 | `GET /admin/api/ad-revenue-report` | admin | [详情](./admin/ad/admin-ad-revenue-report.md)(广告收益报表:分页/场景/`app_env` 筛 + **DAU/ARPU** #120;真实收益侧接穿山甲日表 #92) |
| A13 | `GET / PATCH /admin/api/ad-config` | operator/finance | 广告配置(穿山甲 ID/验签密钥/各场景开关;C 端只读版见 40b)(无单独文档,见 `app/admin/routers/ad_config.py`) |
| A14 | `GET /admin/api/config``PATCH /config/{key}` | operator/finance | 运营可配置项([app_config](../database/app_config.md):奖励常量/提现地板价等;#117 修系统配置下发)(无单独文档,见 `app/admin/routers/config.py`) |
| A16 | `GET / PATCH /admin/api/huawei-review` | operator/tech | 华为审核开关(快速设置权限步能否被用户关闭,落 `app_config.huawei_review`;C 端只读版见 40d)(无单独文档,见 `app/admin/routers/huawei_review.py`) |
| **A·管理员与角色**(super_admin):`GET`/`POST` `/admins``PATCH`/`DELETE` `/admins/{id}`(#126 删除+`pages_override`)、`GET`/`POST` `/roles``GET /roles/catalog``PATCH`/`DELETE` `/roles/{id}`(#117/#126 自定义角色) ||| [列表](./admin/admins/admin-admins-list.md) / [](./admin/admins/admin-admin-create.md) / [改+删](./admin/admins/admin-admin-update.md) / [角色](./admin/admin-roles.md) |
| A15 | `GET /admin/api/audit-logs` | admin | [详情](./admin/admin-audit-logs.md) |
| **A·CPS 运营台**:群/活动 CRUD、`POST /referral-links``POST /orders/reconcile`(美团+京东 #90)、`GET /orders``/stats`、群 `timeseries`/`daily`/`wx-users`/`day-users`(#79) ||| [详情](./admin/admin-cps.md) |
+2 -1
View File
@@ -30,7 +30,8 @@
## 各 tab 行为
- **`rec` 智能推荐**:走【离线库 `meituan_coupon`】筛佣金率 ≥ 3%,`DISTINCT ON(dedup_key)` 去重后按销量降序分页。**纯库查询、不打美团、不依赖 MT 凭证**。库为空(prod 刚部署 / ETL 未跑完)→ `empty`;库异常 → `degraded`。**不显示距离**(库里距离相对城市默认点,对用户无意义)。
- 为何不实时:实测同城热销榜中位佣金 ~0.8%,筛佣金≥3% 后每页剩 0–1 条,既撞 402 又填不满,故从库出;库空时也**不回退实时**(回退同样填不满)。
- **`distance` 距离最近**:外卖搜「外卖」+ 到店搜「美食」(均 `sortField=6`),两路并行顺序翻页,实时按你坐标算距离、由近及远。两路**都失败**且无结果 → `degraded`;否则有结果 `ok` / 无结果 `empty`
- **`distance` 距离最近**:外卖搜「外卖」+ 到店搜「美食」(均 `sortField=6`),两路并行翻页,实时按你坐标算距离、由近及远。两路**都失败**且无结果 → `degraded`;否则有结果 `ok` / 无结果 `empty`
- 翻页成本:美团搜索只能靠 `searchId` 续页,本接口又是无状态的(客户端只传页码)。服务端把沿途 `searchId` 按「量化坐标(~1km)+平台+关键词」缓存 10 分钟(`utils/mt_search_cursor`),**稳态下每翻一页恒定 1 次上游请求**(改前是「取第 N 页 = 发 N 次」);缓存冷/过期才从最近的已知页往后重放。缓存是进程内的,多 worker 不共享 —— 只影响快慢,不影响结果。
- **默认(空 tab)**:逐轮分页的混合 feed(2 外卖 + 1 到店交叉,写死 3 页:爆款 / 今日必推 / 精选+限时),第 4 页返空。两路榜单都失败 → `degraded`
## 错误码
+1
View File
@@ -17,6 +17,7 @@
## 说明
- 从 `meituan_coupon``sale_volume_num` 非空 **且 `city_id` = 反查城市** 的券(#116,同城销量榜),`DISTINCT ON(dedup_key)` 跨源去重(每个「品牌|名|价」只留销量最高一条,同销量再按佣金),按销量降序分页;每页只对当前 ~20 条做 `from_raw` 解析(翻页快,不全表拉取)。
- 分页查询分两步(与 `rec` 共用 `_paged_dedup_ids`):**① 只在 `id + 排序键` 这几个小列上去重/排序/分页,② 再按 id 回表取本页 `raw`**。`raw` 是整条美团原始返回(JSONB,每行数 KB),让它参与排序会把整城数据推过 `work_mem`、落盘做外部归并,而且每翻一页都重来一遍 —— 这是此前「滑到底越来越慢」的主因之一。配套索引 `ix_meituan_coupon_city_dedup_sales` / `..._comm`(见 `alembic/versions/meituan_coupon_feed_indexes.py`)让 `DISTINCT ON` 顺着索引流式去重,免掉排序。
- **不依赖 MT 凭证**(纯库查询)。库为空(prod 刚部署 / ETL 未跑完)→ `status=empty`;库查询异常 → `status=degraded`。均返 `200`、不抛 5xx。
- **仅 PostgreSQL**(`DISTINCT ON` 为 PG 专用)。
+87
View File
@@ -0,0 +1,87 @@
# POST /api/v1/applog/batch — 批量上报客户端运行日志
> 所属:客户端日志组(前缀 `/api/v1/applog` | 鉴权:无(不强制登录,`user_id` 可选带上) | [← 返回 API 索引](../README.md)
批量接收客户端 App 运行日志(自动化步骤 / 网络 / 崩溃 / 调试等),**逐条**封装成单行 JSON 写入独立滚动文件 `logs/app-client.log`(与服务日志 `app-server.log` 分开),由阿里云 Logtail 采进**独立 SLS logstore****不落库**。服务端补 `client_ip`X-Forwarded-For)与 `time`(接收时间,SLS 主时间)。`trace_id` 提到输出行顶层,便于在 SLS 里跨「客户端 / 服务端」两个 logstore 按 trace 拼出端到端链路。**fire-and-forget**:写失败也返回 2xx,不 500。
## 入参
批级公共字段发一次;`logs` 里每条只带日志本身。**每条只有 `client_ts/level/trace_id/tag/msg` 会提到输出顶层,其余自定义字段一律并入输出的 `data`**(防 SLS 索引列爆炸)。
| 字段 | 类型 | 必填 | 说明 |
|---|---|---|---|
| `device_id` | string | ✅(≤64 | 设备 ID(限流 / 分组键) |
| `user_id` | int \| null | ❌ | 登录用户 ID(未登录可空) |
| `app_ver` | string \| null | ❌(≤32 | App 版本 |
| `platform` | string \| null | ❌(≤16 | 平台(`android` / `ios` / `harmony` |
| `sent_at` | int \| null | ❌ | 批次发送时间(epoch ms) |
| `logs` | list[object] | ✅(1-500 条) | 日志数组(每条为对象,内部字段**不强类型**) |
| `logs[].client_ts` | int | ❌ | 端侧日志时间(epoch ms) |
| `logs[].level` | string | ❌ | 级别(服务端归一化大写,截断 ≤16) |
| `logs[].trace_id` | string \| null | ❌ | 关联服务端比价链路的 trace(有服务端交互的日志带上,截断 ≤256) |
| `logs[].tag` | string \| null | ❌ | 模块 / 分类(截断 ≤128) |
| `logs[].msg` | string | ❌ | 消息主体(超 8KB **字节**截断,加 `msg_truncated` 标记) |
| `logs[].*` | any | ❌ | 其余任意自定义字段 → 一律并入输出的 `data`(客户端自带的 `data` 对象会被合并进来) |
Mock 入参:
```json
{
"device_id": "android_abc123def456",
"user_id": 42,
"app_ver": "0.1.5",
"platform": "android",
"sent_at": 1719993700000,
"logs": [
{
"client_ts": 1719993600000,
"level": "info",
"tag": "automation",
"msg": "compare flow start",
"trace_id": "t_ab12cd34"
},
{
"client_ts": 1719993615000,
"level": "error",
"tag": "network",
"msg": "timeout calling /price/step",
"trace_id": "t_ab12cd34",
"http_status": 504,
"retry": 2
}
]
}
```
> 上例第二条的 `http_status` / `retry` 不在白名单 → 会被并入落盘行的 `data`:`{"http_status":504,"retry":2}`
## 出参
响应 `200`:`AppLogIngestOut`
| 字段 | 类型 | 说明 |
|---|---|---|
| `ok` | bool | 固定 `true` |
| `received` | int | 成功写入文件的条数 |
| `dropped` | int | 跳过的条数(服务端处理 / 序列化失败;正常为 `0` |
Mock 出参:
```json
{
"ok": true,
"received": 2,
"dropped": 0
}
```
## 错误码
- `413` 请求体超过上限(默认 2MB;服务端查 `Content-Length`,在 body 校验前拦截。缺该头时由 nginx `client_max_body_size` 兜底)
- `422` `logs` 为空或超过 500 条 / `device_id` 缺失 / 字段类型不符
- `429` 触发限流(同 IP 每分钟 > 120 次)
## 说明
- **落盘 → SLS**:逐条写独立文件 `logs/app-client.log`(单行 JSON`propagate=False` 不污染 `app-server.log`),由 Logtail JSON 模式采进**独立 logstore**;不进数据库。落盘行除白名单字段外,服务端另补 `time`(接收时间)、`source="client"``service``client_ip` 及批级 `device_id/user_id/app_ver/platform/sent_at`
- **`trace_id` 跨层检索**:字段名与服务端日志一致。客户端应给**有服务端交互**的日志带上当初 API 返回的 `trace_id`(如 `/api/v1/price/step` 等透传族返回的 trace),即可在 SLS 里 `trace_id: "xxx"` 一查拼出「客户端视角 + 服务端比价链路」;纯客户端日志不带即可。
- **白名单 + `data` 兜底**:只有 `client_ts/level/trace_id/tag/msg` 上顶层,其余键并入 `data`,把 SLS 索引列钉死在固定集合,防客户端任意 key 撑爆索引。索引字段另有长度上限(`level`≤16 / `trace_id`≤256 / `tag`≤128 / `msg`≤8KB)。
- **fire-and-forget**:写文件失败也返回 2xx(避免客户端重试风暴);网络重试可能在 SLS 造成重复条目,可接受。
- **量级建议**:客户端做等级过滤 / 采样,攒到一定量再批量上报;单批 ≤500 条、body ≤2MB。
- `client_ts` 是端侧时间(客户端时钟不可信),`time` 由服务端补(可靠时间轴)。
- `user_id` 不靠 JWT:未登录态也要采日志;带上便于按用户排查。
+5 -1
View File
@@ -42,8 +42,12 @@
## 索引与约束
- 唯一约束 `uq_meituan_coupon_source_sign`(`source`, `product_view_sign`)—— ETL upsert 的冲突目标。
- 单列 index:`source``city_id``brand_name``sale_volume_num``commission_percent``dedup_key``last_seen`
- 复合 index(`meituan_coupon_feed_indexes` 迁移,2026-07-23 加):
- `ix_meituan_coupon_city_dedup_sales`(`city_id`, `dedup_key`, `sale_volume_num DESC`, `commission_percent DESC`)——「销量最高」。
- `ix_meituan_coupon_city_dedup_comm`(`city_id`, `dedup_key`, `commission_percent DESC`)——「智能推荐」。
- 列顺序对齐 `WHERE city_id=? … ORDER BY dedup_key, <排序键> DESC`,让 `DISTINCT ON` 顺着索引流式去重、免掉排序。
- **字段核对(2026-06-10)**:模型 ↔ 迁移(`meituan_coupon_table`)↔ ETL 解析三处一致;解析截断宽度均 ≤ 列宽(name256 / brand128 / poi128 / head512 / sign128,`dedup_key` = md5 32 字符 ≤ 64)。
- **索引评估**:`rec` / `sales` 查询的理想索引是复合 `(dedup_key, sale_volume_num DESC)``(dedup_key, commission_percent DESC)`(匹配 `DISTINCT ON`);但当前**单城几千行,seq scan 即亚毫秒**,现有单列索引已够用,复合索引留作多城 / 放量时再加,避免过早优化
- **索引评估修订(2026-07-23)**:此前判断「单城几千行 seq scan 即亚毫秒,复合索引留作放量时再加」——**漏算了 `raw`**。原查询用 `select(MeituanCoupon)` 做子查询,`raw`(JSONB,每行数 KB)被一起拖进 `DISTINCT ON` 与分页两轮排序,整城体量轻松推过 `work_mem` → 落盘外部归并,且**每翻一页重来一次**,是首页「滑到底越来越慢」的主因之一。现已双管齐下:接口侧改成「先在小列上排出本页 id,再按 id 回表取 `raw`」(`api/v1/meituan.py``_paged_dedup_ids`),库侧补上上面两条复合索引
## 过期清理策略
- ETL 每轮(每小时)末尾按 **`last_seen` 的 TTL** 清理:`DELETE WHERE last_seen < now() - prune_hours`(默认 24h,`--prune-hours` 可调,0=不清)。
@@ -0,0 +1,224 @@
CheckSmsVerifyCode - 核验验证码
更新时间:2026年3月19日 20:02:53
核验短信验证码并返回核验是否成功的结果。
调试
您可以在OpenAPI Explorer中直接运行该接口,免去您计算签名的困扰。运行成功后,OpenAPI Explorer可以自动生成SDK代码示例。
调试
授权信息
下表是API对应的授权信息,可以在RAM权限策略语句的Action元素中使用,用来给RAM用户或RAM角色授予调用此API的权限。具体说明如下:
操作:是指具体的权限点。
访问级别:是指每个操作的访问级别,取值为写入(Write)、读取(Read)或列出(List)。
资源类型:是指操作中支持授权的资源类型。具体说明如下:
对于必选的资源类型,用前面加 * 表示。
对于不支持资源级授权的操作,用全部资源表示。
条件关键字:是指云产品自身定义的条件关键字。
关联操作:是指成功执行操作所需要的其他权限。操作者必须同时具备关联操作的权限,操作才能成功。
放大查看
操作
访问级别
资源类型
条件关键字
关联操作
dypns:CheckSmsVerifyCode
none
*全部资源
*
无 无
请求参数
放大查看
名称
类型
必填
描述
示例值
SchemeName
string
方案名称,如果不填则为“默认方案”。最多不超过 20 个字符。
重要 如果发送接口的方案名称不为空,请确保该参数不为空且与发送接口的方案名称参数一致
测试方案
CountryCode
string
号码国家编码,默认为 86。
86
PhoneNumber
string
手机号。
186****0000
OutId
string
外部流水号。
12123231
VerifyCode
string
验证码。
说明
SendSmsVerifyCode 接口的字段 TemplateParam,配置方式有 2 种:
{"code":"##code##","min":"5"}
{"code":"123456","min":"5"}
{"code":"##code##","min":"5"}验证码是 api 动态生成的,阿里云接口可以完成校验。
{"code":"123456","min":"5"}验证码是用户配置的不是 api 动态生成,阿里云接口无法校验。
请您按照实际情况传入对应的验证码。
1231
CaseAuthPolicy
integer
验证码大小写字母核验策略。取值:
1:不区分大小写。
2:区分大小写。
1
返回参数
放大查看
名称
类型
描述
示例值
object
AccessDeniedDetail
string
访问被拒绝详细信息。
Message
string
状态码的描述。
成功
Model
object
请求结果数据。
OutId
string
外部流水号。
1212312
VerifyResult
string
短信验证码核验结果。取值:
PASS:短信验证码核验成功。
UNKNOWN:短信验证码核验失败。
PASS
Code
string
接口请求状态码。
返回 OK 代表请求成功。
其他错误码,请参见返回码。
重要 接口请求成功不代表短信验证码核验成功,短信验证码核验结果仅以Model.VerifyResult参数返回值为准。
OK
Success
boolean
接口调用是否成功。取值:
true:接口调用成功。
false:接口调用失败。
重要 接口调用成功不代表短信验证码核验成功,短信验证码核验结果仅以Model.VerifyResult参数返回值为准。
true
RequestId
string
CF8854E5-DB21-3E5D-A9B1-DDC752FD7384
示例
正常返回示例
JSON格式
放大查看复制代码
{
"AccessDeniedDetail": "无",
"Message": "成功",
"Model": {
"OutId": "1212312",
"VerifyResult": "PASS"
},
"Code": "OK",
"Success": true,
"RequestId": "CF8854E5-DB21-3E5D-A9B1-DDC752FD7384"
}
@@ -0,0 +1,396 @@
SendSmsVerifyCode - 发送短信验证码
更新时间:2026年7月3日 09:54:53
发送短信验证码。
接口说明
由于运营商近期加强对短信签名的管控。您自定义的签名面临下发失败问题,推荐您使用号码认证控制台赠送的短信签名和模板进行短信认证。系统赠送签名必须搭配系统赠送模板使用。
请确保在使用该接口前,已充分了解号码认证服务产品的收费方式和价格,短信认证服务仅收取短信发送费用(按运营商回执状态计费,短信提交成功但运营商回执失败时不计费),核验服务免费。
调试
您可以在OpenAPI Explorer中直接运行该接口,免去您计算签名的困扰。运行成功后,OpenAPI Explorer可以自动生成SDK代码示例。
调试
授权信息
下表是API对应的授权信息,可以在RAM权限策略语句的Action元素中使用,用来给RAM用户或RAM角色授予调用此API的权限。具体说明如下:
操作:是指具体的权限点。
访问级别:是指每个操作的访问级别,取值为写入(Write)、读取(Read)或列出(List)。
资源类型:是指操作中支持授权的资源类型。具体说明如下:
对于必选的资源类型,用前面加 * 表示。
对于不支持资源级授权的操作,用全部资源表示。
条件关键字:是指云产品自身定义的条件关键字。
关联操作:是指成功执行操作所需要的其他权限。操作者必须同时具备关联操作的权限,操作才能成功。
放大查看
操作
访问级别
资源类型
条件关键字
关联操作
dypns:SendSmsVerifyCode
create
*全部资源
*
无 无
请求参数
放大查看
名称
类型
必填
描述
示例值
SchemeName
string
方案名称,如果不填则为“默认方案”。最多不超过 20 个字符。
测试方案
CountryCode
string
号码国家编码。默认为 86,目前也仅支持国内号码发送。
86
PhoneNumber
string
短信接收方手机号。
130****0000
SignName
string
签名名称。暂不支持使用自定义签名,请使用系统赠送的签名,您可在赠送签名配置页面选择需要下发的签名。
恒创联众
TemplateCode
string
短信模板 CODE。参数SignName选择赠送签名时,必须搭配赠送模板下发短信。您可在赠送模板配置页面选择适用您业务场景的模板。
100001
TemplateParam
string
短信模板参数。验证码位置有两种传值方式:
可使用"##code##"替代,由参数 CodeType 指定验证码生成规则;
也可直接传入具体的验证码值,直接下发至接收方。
示例:如模板内容为:“您的验证码是${code},有效期${min}分钟,请勿告诉他人。”。
重要 上文中的 code 请替换成您实际申请的验证码模板中的参数名称
该字段可传入{"code":"##code##","min":"5"}由系统根据规则生成验证码;
或直接传入指定的验证码值{"code":"123456","min":"5"}。
说明
{"code":"##code##","min":"5"}验证码是 api 动态生成的,阿里云接口可以完成校验。
{"code":"123456","min":"5"}验证码是用户配置的不是 api 动态生成,阿里云接口无法校验。
说明
如果 JSON 中需要带换行符,请参照标准的 JSON 协议处理。
模板变量规范,请参见短信模板规范。
{"code":"##code##","min":"5"}
SmsUpExtendCode
string
上行短信扩展码。上行短信指发送给通信服务提供商的短信,用于定制某种服务、完成查询,或是办理某种业务等,需要收费,按运营商普通短信资费进行扣费。
说明
扩展码是生成签名时系统自动默认生成的,不支持自行传入。无特殊需要此字段的用户请忽略此字段。如需使用,请联系您的商务经理。
1213123
OutId
string
外部流水号。
外部流水号(透传)
CodeLength
integer
验证码长度支持 4~8 位长度,默认是 4 位。
4
ValidTime
integer
验证码有效时长,单位秒,默认为 300 秒。
300
DuplicatePolicy
integer
核验规则,当有效时间内对同场景内的同号码重复发送验证码时,旧验证码如何处理。
1:覆盖处理(默认),即旧验证码会失效掉。
2:保留,即多个验证码都是在有效期内都可以校验通过。
枚举值:
1 :
覆盖
2 :
保留
1
Interval
integer
时间间隔,单位:秒。即多久间隔可以发送一次验证码,用于频控,默认 60 秒。
60
CodeType
integer
生成的验证码类型。当参数 TemplateParam 传入占位符时,此参数必填,将由系统根据指定的规则生成验证码。取值:
1:纯数字(默认)。
2:纯大写字母。
3:纯小写字母。
4:大小字母混合。
5:数字+大写字母混合。
6:数字+小写字母混合。
7:数字+大小写字母混合。
枚举值:
1 :
纯数字
2 :
纯大写字母
3 :
纯小写字母
4 :
大小字母混合
5 :
数字+大写字母混合
6 :
数字+小写字母混合
7 :
数字+大小写字母混合
1
ReturnVerifyCode
boolean
是否返回验证码。取值:
true:返回。
false:不返回。
true
AutoRetry
integer
是否自动替换签名重试(默认开启),可取值:
1 开启自动重试功能,开启后,在验证码有效期内,当运营商返回明确的失败状态时,允许阿里云尽可能的尝试使用其他方式发送验证码,以提升发送成功率。其他方式包括且不限于:通过其他运营商重试、更换签名重试等
0 不开启自动重试
是否自动重试
返回参数
放大查看
名称
类型
描述
示例值
object
AccessDeniedDetail
string
访问被拒绝详细信息。
Message
string
状态码的描述。
成功
RequestId
string
请求 ID。
CC3BB6D2-2FDF-4321-9DCE-B38165CE4C47
Model
object
请求结果数据。
VerifyCode
string
验证码。
4232
RequestId
string
请求 ID。
a3671ccf-0102-4c8e-8797-a3678e091d09
OutId
string
外部流水号。
1231231313
BizId
string
业务 ID。
112231421412414124123^4
Code
string
请求状态码。返回 OK 代表请求成功。其他错误码,请参见返回码列表。
OK
Success
boolean
请求是否成功。
true:请求成功。
false:请求失败。
true
示例
正常返回示例
JSON格式
放大查看复制代码
{
"AccessDeniedDetail": "无",
"Message": "成功 ",
"RequestId": "CC3BB6D2-2FDF-4321-9DCE-B38165CE4C47",
"Model": {
"VerifyCode": "4232",
"RequestId": "a3671ccf-0102-4c8e-8797-a3678e091d09",
"OutId": "1231231313",
"BizId": "112231421412414124123^4"
},
"Code": "OK",
"Success": true
}
错误码
放大查看
HTTP status code
错误码
错误信息
描述
400 MOBILE_NUMBER_ILLEGAL The mobile number is illegal. 手机号码格式错误
400 BUSINESS_LIMIT_CONTROL The number has exceeded the limit for the day. 触发号码天级流控
400 FREQUENCY_FAIL Check frequency fail. 频控校验未通过
400 INVALID_PARAMETERS parameter is not valid. 非法参数
400 FUNCTION_NOT_OPENED You have not opened this function. 没有开通融合认证功能
+117
View File
@@ -0,0 +1,117 @@
# 创蓝云智(253/蓝创云智)模板短信 v2 发送接口
> 官方文档:<https://doc.chuanglan.com/document/HAQYSZKH9HT5Z50L>
> 用途:手机号 + 验证码登录的**验证码短信下发**(本服务生成码 → 创蓝只负责发送,属自管码 Mode B,与极光同模式)。
> 本文件为**接口调研摘要**,供 `app/integrations/sms/chuanglan.py` 实现对照。以线上文档为准。
## 接口概览
| 项 | 值 |
|---|---|
| 请求地址 | `POST https://smssh.253.com/msg/sms/v2/tpl/send` |
| Content-Type | `application/json`UTF-8 |
| 协议 | HTTPS |
| 鉴权 | HMAC-SHA256 签名头 `X-QA-Hmac-Signature`(推荐)**或** body 明文 `password`(二选一) |
## 鉴权:两种方式(二选一,不可并用)
1. **HMAC 签名头(推荐,密码不上行)**:请求头带 `X-QA-Hmac-Signature`body **不放** `password`
2. **明文密码**body 放 `password`,不带签名头。
### HMAC-SHA256 签名算法
1. `md5Password = MD5(password)` —— 32 位**小写十六进制**。
2. 取三个值 `[md5Password, timestamp, nonce]`,**按字典序升序排序**,**无分隔符拼接**,再 `replaceAll("\\s+", "")` 去除所有空白。
3. `signature = HmacSHA256(key = md5Password, message = 上一步拼接串)` —— 输出**小写十六进制**。
4. 放入请求头:`X-QA-Hmac-Signature: <signature>`
> 注意 `key` 就是 `md5Password` 本身(32 位 hex 字符串),不是原始 password。`timestamp` / `nonce` 同时也是 body 字段,必须与签名里用的一致。
## 请求参数(bodyJSON
| 参数 | 类型 | 必填 | 说明 |
|---|---|---|---|
| `account` | String | 是 | API 账号;验证码短信用 **`YZM` 前缀**账号(如 `YZM0000001` |
| `timestamp` | String | 是 | Unix 秒级时间戳;**60 秒**内有效,过期报 139 |
| `nonce` | String | 是 | 32 位随机串(防重放) |
| `phoneNumbers` | String | 是 | 手机号,逗号分隔最多 1000 个;**YZM 验证码账号不支持批量,只能单号** |
| `templateId` | String | 是 | 模板 ID(控制台创建 / 模板接口查询) |
| `templateParamJson` | String | 条件 | 模板变量,JSON 字符串;模板有 `{s}` 占位符时必填(见下) |
| `password` | String | 条件 | 仅在**不使用**签名头时放 body |
| `signature` | String | 条件 | **短信签名文案**(如 `【创蓝云智】`);模板未关联签名时必填。**注意与鉴权头 `X-QA-Hmac-Signature` 是两回事** |
| `report` | String | 否 | `"true"` 时接收状态回执 |
| `callbackUrl` | String | 否 | 回执回调完整 URL |
| `uid` | String | 否 | 自定义标识(≤256 字符),回执原样返回 |
| `extend` | String | 否 | 数字扩展码(≤5 位),用于上行匹配 |
### `templateParamJson` 格式与 `{s}` 占位
- 模板内容用 `{s}` 作占位符,例:`您正在申请手机注册,验证码为:{s},5分钟内有效!`
- `templateParamJson`**JSON 数组,元素为对象**,键按 `param1``param2`…递增;第 1 个 `{s}``param1`,第 2 个 ← `param2`
- 单条验证码(一个 `{s}` = 验证码)示例:`"[{\"param1\":\"123456\"}]"`
## 响应格式
```json
{
"code": "000000",
"msgId": "25071018345400902898000000000001",
"time": "20250710183454",
"successNum": "1",
"failNum": "0",
"errorMsg": ""
}
```
| 字段 | 说明 |
|---|---|
| `code` | 状态码,`"000000"` = 成功 |
| `msgId` | 消息 ID32 位) |
| `time` | 响应时间戳 |
| `successNum` / `failNum` | 提交成功 / 失败条数 |
| `errorMsg` | 错误描述(成功为空) |
## 响应 / 错误码(节选)
| code | 含义 | 归属 |
|---|---|---|
| `000000` | 成功 | — |
| `101` | 账号不存在 | 客服 |
| `102` | 密码错误 | 客服 |
| `103` | 提交速度过快(超频) | 客服 |
| `107` | 手机号码错误 | 客服 |
| `109` | 无发送量(余额/套餐不足) | 销售 |
| `110` | 不在发送时段 | 销售 |
| `116` | 签名不合法 / 未带签名 | 服务 |
| `117` | IP 未加白名单 | 服务 |
| `118` | 账号无发送权限 | 服务 |
| `124` | 模板内容不匹配 | 服务 |
| `129` | JSON 格式错误 | 客服 |
| `135` | 相同手机号内容重复 | 客服 |
| `139` | 时间戳过期 | 客服 |
| `152` | 模板不存在 | 服务 |
| `158` | 退订文案不合规 | 客服 |
## 完整请求示例(单条验证码,明文密码方式省略 password 用签名头)
```json
{
"account": "YZM0000001",
"timestamp": "1752143733",
"nonce": "x4zfk0y5foqwx6cbnw3bfmimy98abqs1",
"phoneNumbers": "17601337176",
"templateId": "1021143438",
"templateParamJson": "[{\"param1\":\"123456\"}]",
"report": "true"
}
```
HMAC 方式:另加请求头 `X-QA-Hmac-Signature: <算法输出>`body 不含 `password`。)
## 接入要点
- **IP 白名单**:服务器出网 IP 必须在控制台加白,否则 117。
- **验证码账号(YZM**:无发送时段限制;单号发送、不支持批量。
- **时间戳 60s**`timestamp` 与本地时钟偏差过大会 139,注意服务器时间同步。
- **退订文案**:仅支持 `拒收请回复R`,且必须在短信末尾(验证码短信一般无需)。
- **签名 vs 鉴权头**`signature`body= 短信开头的 `【品牌】` 文案;`X-QA-Hmac-Signature`(header)= 请求鉴权。二者含义完全不同,勿混。
+19 -2
View File
@@ -1,9 +1,26 @@
# 短信验证码(sms
> 文件:`app/integrations/sms.py` | 关联接口:[auth-sms-send](../api/auth-sms-send.md) · [auth-sms-login](../api/auth-sms-login.md) | [← 集成索引](./README.md)
> 文件:`app/integrations/sms/`(分派器 `__init__` + `jiguang` / `aliyun` provider + `base`) | 关联接口:[auth-sms-send](../api/auth-sms-send.md) · [auth-sms-login](../api/auth-sms-login.md) | [← 集成索引](./README.md)
## 作用
手机号 + 验证码登录的验证码发送 / 校验。**已接极光短信 REST**,由 `SMS_MOCK` 切 mock / real。
手机号 + 验证码登录的验证码发送 / 校验。支持**可切换 provider**(`SMS_PROVIDER`):`jiguang`(默认,极光自管码)/ `aliyun`(阿里云号码认证托管码)/ `chuanglan`(创蓝云智模板短信,自管码)。`SMS_MOCK` 切 mock / real。
## 短信提供商(`SMS_PROVIDER`,可切换 + 灰度回退)
| | `jiguang`(默认) | `aliyun` | `chuanglan` |
|---|---|---|---|
| 验证码模式 | 自管码 Mode B | 托管码 Mode A | 自管码 Mode B(与极光同) |
| 验证码 | **本服务生成**、极光只下发、**本地内存校验** | **阿里云生成 + 下发 + 校验**(dypns 号码认证,核验免费) | **本服务生成**、创蓝只下发、**本地内存校验** |
| 发码 | 极光 `/v1/messages` | `SendSmsVerifyCode`(`##code##` 占位) | 创蓝 `tpl/send` v2(HMAC 签名头,password 不上行) |
| 校验 | 比对本地存码 | `CheckSmsVerifyCode``PASS` / `UNKNOWN` | 比对本地存码 |
| 多 worker | ⚠️ 内存存码不共享(见已知局限) | ✅ 阿里云托管,天然共享 | ⚠️ 内存存码不共享(与极光同级债) |
| 防爆破 | 单码失败 `SMS_MAX_VERIFY_ATTEMPTS` 次即作废 | **同语义**(本地 per-phone 失败计数) | **同语义**(复制自极光) |
| 单号冷却 | 本地 `SMS_SEND_INTERVAL_SEC` | 交给阿里云 `Interval` | 本地 `SMS_SEND_INTERVAL_SEC` |
| 依赖 | `httpx`(复用) | SDK `alibabacloud_dypnsapi20170525` | `httpx` + 标准库 `hashlib`/`hmac`(**无新依赖**) |
- `aliyun` 需在**号码认证控制台开通「融合认证」**,用系统赠送签名 + 赠送模板;配置见 `.env.example``ALIYUN_SMS_*`,SDK 为 `alibabacloud_dypnsapi20170525`
- `chuanglan`**YZM 前缀验证码账号**,服务器出网 IP 须在创蓝控制台**加白名单**(否则 117);Mode B 存码/冷却/校验逻辑是**从极光隔离复制**(极光文件不动),仅发送走 HMAC 签名。配置见 `.env.example``CHUANGLAN_SMS_*`,接口调研见 [chuanglan/tpl-send.md](chuanglan/tpl-send.md),设计见 [spec](../superpowers/specs/2026-07-26-chuanglan-sms-verify-design.md)。
**以下章节描述 `jiguang` provider(自管码)细节**(`chuanglan` 的存码/冷却/校验语义与之相同)。
| | mock(`SMS_MOCK=true`,默认 / 开发测试) | real(`SMS_MOCK=false`,生产) |
|---|---|---|
@@ -0,0 +1,561 @@
# 客户端运行日志批量上报 Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** 新增 `POST /api/v1/applog/batch`,把客户端批量上报的运行日志逐条写进独立滚动文件 `logs/app-client.log`,供 Aliyun Logtail 采进独立 SLS logstore。
**Architecture:** 复用两个现成范式——analytics 的「批量入口 + 服务端补 IP」+ `app-server.log` 的「单行 JSON + RotatingFileHandler + Logtail」。核心逻辑在写入层 `app/core/client_log.py`:专用 `client` logger`propagate=False`,不污染服务日志),每条按「白名单键提顶层 + 其余并入 `data`」封装成一行 JSON`trace_id` 提到顶层以便跨层检索。端点瘦、fire-and-forget(写失败不 500)。
**Tech Stack:** FastAPI、Pydantic v2、Python `logging.handlers.RotatingFileHandler`、pytest + `TestClient`
**Spec:** [docs/superpowers/specs/2026-07-19-client-applog-ingest-design.md](../specs/2026-07-19-client-applog-ingest-design.md)
---
## File Structure
| 文件 | 职责 | 动作 |
|---|---|---|
| `app/core/client_log.py` | 专用 logger/handler、滚动配置、白名单+`data` 兜底与信封拼装、`write_records()` | Create |
| `app/schemas/applog.py` | `AppLogBatchIn``logs: list[dict]`/ `AppLogIngestOut` | Create |
| `app/api/v1/applog.py` | 瘦路由:body 上限依赖 → 调 writer → 返回计数 | Create |
| `app/main.py` | 注册 `applog_router` | Modify |
| `tests/test_applog.py` | writer 单测 + 端点集成测试 | Create |
前置约束(写进代码注释):`RotatingFileHandler` 多进程并发 `doRollover()` 会损坏/丢日志;本方案依赖生产 `--workers 1`(与限流器/SMS 码/SQLite 同一既有假设)。扩 worker 前必须换 `QueueHandler`→单写入者 / 外部 logrotate / 写 stdout 交 journald。
---
## Task 1: 专用落盘 writer `app/core/client_log.py`
**Files:**
- Create: `app/core/client_log.py`
- Test: `tests/test_applog.py`
- [ ] **Step 1: 写失败测试(writer 层)**
创建 `tests/test_applog.py`
```python
"""客户端运行日志上报:writer 单测 + 端点集成测试。"""
from __future__ import annotations
import json
from pathlib import Path
import pytest
from app.core import client_log
@pytest.fixture()
def client_log_file(tmp_path, monkeypatch):
"""把客户端日志切到临时文件,并重置 writer 单例使其按当时 env 重建。"""
p = tmp_path / "app-client.log"
monkeypatch.setenv("CLIENT_LOG_FILE", str(p))
client_log.reset_client_logger()
yield p
client_log.reset_client_logger()
def _read_lines(p: Path) -> list[dict]:
text = p.read_text(encoding="utf-8").strip()
return [json.loads(ln) for ln in text.splitlines() if ln]
# ---------------- writer 层 ----------------
def _meta(**kw) -> dict:
base = {"device_id": "d-1", "user_id": None, "app_ver": None,
"platform": None, "sent_at": None}
base.update(kw)
return base
def test_writer_writes_one_line_per_record(client_log_file):
recs = [
{"client_ts": 1737000000000, "level": "info", "msg": "hello"},
{"client_ts": 1737000000001, "level": "error", "msg": "boom", "tag": "net"},
]
received, dropped = client_log.write_records(
recs, meta=_meta(user_id=42, app_ver="1.2.3", platform="android"),
client_ip="1.2.3.4",
)
assert (received, dropped) == (2, 0)
lines = _read_lines(client_log_file)
assert len(lines) == 2
assert lines[0]["source"] == "client"
assert lines[0]["service"] == "app-client"
assert lines[0]["device_id"] == "d-1"
assert lines[0]["user_id"] == 42
assert lines[0]["app_ver"] == "1.2.3"
assert lines[0]["client_ip"] == "1.2.3.4"
assert lines[0]["level"] == "INFO" # 归一化大写
assert lines[0]["msg"] == "hello"
assert lines[0]["client_ts"] == 1737000000000
assert lines[1]["tag"] == "net"
def test_writer_hoists_trace_id_to_top_level(client_log_file):
client_log.write_records(
[{"client_ts": 1, "level": "info", "msg": "x", "trace_id": "abc123"}],
meta=_meta(), client_ip="",
)
assert _read_lines(client_log_file)[0]["trace_id"] == "abc123"
def test_writer_sweeps_unknown_keys_into_data(client_log_file):
client_log.write_records(
[{"client_ts": 1, "level": "info", "msg": "x",
"foo": 123, "data": {"bar": "baz"}}],
meta=_meta(), client_ip="",
)
line = _read_lines(client_log_file)[0]
assert "foo" not in line # 白名单外不进顶层
assert line["data"]["foo"] == 123 # 兜底进 data
assert line["data"]["bar"] == "baz" # 客户端自带 data 合并进来
def test_writer_truncates_oversize_msg(client_log_file, monkeypatch):
monkeypatch.setenv("APPLOG_MAX_MSG_BYTES", "10")
received, dropped = client_log.write_records(
[{"client_ts": 1, "level": "info", "msg": "x" * 100}],
meta=_meta(), client_ip="",
)
assert (received, dropped) == (1, 0) # 截断而非丢弃
line = _read_lines(client_log_file)[0]
assert line["msg_truncated"] is True
assert line["msg"].endswith("…[truncated]")
def test_writer_logger_does_not_propagate(client_log_file):
lg = client_log.get_logger()
assert lg.name == "client"
assert lg.propagate is False # 不冒泡到 root → 不写 app-server.log
```
- [ ] **Step 2: 跑测试确认失败**
Run: `pytest tests/test_applog.py -q`
Expected: FAIL —— `AttributeError: module 'app.core.client_log' has no attribute 'reset_client_logger'`(模块尚不存在)。
- [ ] **Step 3: 实现 `app/core/client_log.py`**
```python
"""客户端运行日志专用落盘 writer(独立于服务端 app-server.log)。
- 独占 logger "client" + 自己的 RotatingFileHandler,propagate=False → 不污染 app-server.log。
- 每条按「白名单键(client_ts/level/trace_id/tag/msg)提顶层 + 其余并入 data」封装,再
json.dumps 成一行写出(钉死 SLS 索引列;见 spec §5)。formatter 用 %(message)s——行本身
已是 JSON,不能再过 JsonFormatter 二次编码。
- 滚动 20MB×10(env 可调),与服务日志同机制。
⚠️ 依赖 --workers 1:RotatingFileHandler 多进程并发 doRollover 会损坏/丢日志;扩 worker
前换 QueueHandler→单写入者 / 外部 logrotate(copytruncate) / 写 stdout 交 journald。
服务端补的字段(time/source/service/client_ip/device_id/...)是「事实」,与客户端自述分开。
`time` 用服务端接收时间作 SLS 主时间(客户端时钟不可信),client_ts 另存为可查字段。
"""
from __future__ import annotations
import json
import logging
import os
from datetime import datetime
from logging.handlers import RotatingFileHandler
from pathlib import Path
# 仅这些客户端键提到输出行顶层;其余(含客户端自带 data)一律并入 data,防 SLS 索引列爆炸
_TOP_LEVEL_KEYS = ("client_ts", "level", "trace_id", "tag", "msg")
_logger: logging.Logger | None = None
def _max_msg_bytes() -> int:
return int(os.getenv("APPLOG_MAX_MSG_BYTES", "8192"))
def _build_logger() -> logging.Logger:
lg = logging.getLogger("client")
lg.setLevel(logging.INFO)
lg.propagate = False # 不冒泡到 root → 不写进 app-server.log
log_file = os.getenv("CLIENT_LOG_FILE") or str(
Path(os.getenv("LOG_DIR", "logs")) / "app-client.log"
)
Path(log_file).parent.mkdir(parents=True, exist_ok=True)
handler = RotatingFileHandler(
log_file,
maxBytes=int(os.getenv("CLIENT_LOG_MAX_BYTES", str(20 * 1024 * 1024))),
backupCount=int(os.getenv("CLIENT_LOG_BACKUP_COUNT", "10")),
encoding="utf-8",
)
handler.setFormatter(logging.Formatter("%(message)s")) # 行已是 JSON,不再包装
lg.handlers = [handler]
return lg
def get_logger() -> logging.Logger:
global _logger
if _logger is None:
_logger = _build_logger()
return _logger
def reset_client_logger() -> None:
"""测试用:关闭并丢弃当前 logger,使下次 get_logger 按当时 env 重建(切临时文件)。"""
global _logger
if _logger is not None:
for h in list(_logger.handlers):
h.close()
_logger.handlers = []
_logger = None
def _truncate_msg(msg: str) -> tuple[str, bool]:
raw = msg.encode("utf-8")
limit = _max_msg_bytes()
if len(raw) <= limit:
return msg, False
# 按字节截断后解码,忽略截断处半个多字节字符
return raw[:limit].decode("utf-8", "ignore") + "…[truncated]", True
def _build_line(
record: dict, *, meta: dict, client_ip: str, service: str, now_iso: str
) -> str:
out: dict = {
"time": now_iso,
"source": "client",
"service": service,
"client_ip": client_ip,
}
# 批级公共字段(非空才带)
for k in ("device_id", "user_id", "app_ver", "platform", "sent_at"):
v = meta.get(k)
if v is not None:
out[k] = v
# 白名单键提顶层
if record.get("level") is not None:
out["level"] = str(record["level"]).upper()
if record.get("trace_id"):
out["trace_id"] = record["trace_id"]
if record.get("tag"):
out["tag"] = record["tag"]
if record.get("client_ts") is not None:
out["client_ts"] = record["client_ts"]
if record.get("msg") is not None:
msg, truncated = _truncate_msg(str(record["msg"]))
out["msg"] = msg
if truncated:
out["msg_truncated"] = True
# 其余键(含客户端自带 data)并入 data
data: dict = {}
client_data = record.get("data")
if isinstance(client_data, dict):
data.update(client_data)
for k, v in record.items():
if k in _TOP_LEVEL_KEYS or k == "data":
continue
data[k] = v
if data:
out["data"] = data
return json.dumps(out, ensure_ascii=False, default=str)
def write_records(
records: list[dict], *, meta: dict, client_ip: str
) -> tuple[int, int]:
"""把一批客户端日志逐条写入专用文件。返回 (received, dropped)。
尽力而为(fire-and-forget):logger 初始化或单条写入失败只跳过并计 dropped,
不抛给上层——端点因此永不因写日志而 500。
"""
try:
lg = get_logger()
except Exception: # noqa: BLE001 — 初始化失败也不能让端点 500
logging.getLogger("shagua.applog").exception("client log writer init failed")
return 0, len(records)
service = os.getenv("CLIENT_LOG_SERVICE_NAME", "app-client")
now_iso = datetime.now().strftime("%Y-%m-%dT%H:%M:%S.%f")[:-3]
received = dropped = 0
for rec in records:
try:
line = _build_line(
rec, meta=meta, client_ip=client_ip, service=service, now_iso=now_iso
)
lg.info(line)
received += 1
except Exception: # noqa: BLE001 — 坏条跳过,不影响其余
dropped += 1
return received, dropped
```
- [ ] **Step 4: 跑测试确认通过**
Run: `pytest tests/test_applog.py -q`
Expected: PASS5 个 writer 测试全绿)。
- [ ] **Step 5: ruff**
Run: `ruff check app/core/client_log.py tests/test_applog.py`
Expected: 无错误(如有 import 排序等自动可修问题:`ruff check --fix`)。
- [ ] **Step 6: 提交**
```bash
git add app/core/client_log.py tests/test_applog.py
git commit -m "feat(applog): 客户端日志专用落盘 writer(白名单+data 兜底, propagate=False)"
```
---
## Task 2: 端点 + schema + 注册
**Files:**
- Create: `app/schemas/applog.py`
- Create: `app/api/v1/applog.py`
- Modify: `app/main.py`import + `include_router`
- Test: `tests/test_applog.py`(追加端点用例)
- [ ] **Step 1: 追加失败测试(端点层)**
`tests/test_applog.py` 末尾追加:
```python
# ---------------- 端点层 ----------------
def _post(client, body):
return client.post("/api/v1/applog/batch", json=body)
def test_endpoint_happy_path(client, client_log_file):
body = {
"device_id": "d-1", "user_id": 42, "app_ver": "1.2.3", "platform": "android",
"logs": [
{"client_ts": 1, "level": "info", "msg": "a", "trace_id": "t1"},
{"client_ts": 2, "level": "warn", "msg": "b"},
],
}
resp = _post(client, body)
assert resp.status_code == 200
assert resp.json() == {"ok": True, "received": 2, "dropped": 0}
lines = _read_lines(client_log_file)
assert len(lines) == 2
assert lines[0]["trace_id"] == "t1" # 端到端:trace_id 落到文件顶层
assert lines[0]["client_ip"] # 服务端补了 IP
def test_endpoint_rejects_batch_over_max(client, client_log_file):
body = {"device_id": "d-1",
"logs": [{"client_ts": i, "level": "info", "msg": str(i)} for i in range(501)]}
resp = _post(client, body)
assert resp.status_code == 422 # Pydantic max_length=500
def test_endpoint_rejects_body_over_limit(client, client_log_file, monkeypatch):
monkeypatch.setenv("APPLOG_MAX_BODY_BYTES", "50")
body = {"device_id": "d-1", "logs": [{"client_ts": 1, "level": "info", "msg": "x" * 500}]}
resp = _post(client, body)
assert resp.status_code == 413 # 依赖查 Content-Length,body 校验前拦截
def test_endpoint_returns_200_on_write_failure(client, client_log_file, monkeypatch):
class _BoomLogger:
name = "client"
propagate = False
handlers: list = []
def info(self, *a, **k):
raise RuntimeError("disk full")
monkeypatch.setattr(client_log, "get_logger", lambda: _BoomLogger())
body = {"device_id": "d-1", "logs": [
{"client_ts": 1, "level": "info", "msg": "a"},
{"client_ts": 2, "level": "info", "msg": "b"}]}
resp = _post(client, body)
assert resp.status_code == 200 # fire-and-forget:写失败不 500
assert resp.json() == {"ok": True, "received": 0, "dropped": 2}
def test_endpoint_missing_device_id_is_422(client, client_log_file):
resp = _post(client, {"logs": [{"client_ts": 1, "level": "info", "msg": "a"}]})
assert resp.status_code == 422 # device_id 必填
```
- [ ] **Step 2: 跑测试确认失败**
Run: `pytest tests/test_applog.py -q`
Expected: FAIL —— 端点未注册,`POST /api/v1/applog/batch` 返回 404happy-path 断言 200 失败)。
- [ ] **Step 3: 实现 schema `app/schemas/applog.py`**
```python
"""客户端运行日志批量上报 schema。
批级公共字段(device_id/user_id/app_ver/platform/sent_at)发一次;logs 为原始 dict 列表,
**不强类型**——尽力而为的日志链路,单条内容异常不该让整批 422。每条的「白名单键 + data
兜底」拆分在写入层 [app.core.client_log] 做(见 spec §4/§5)。
"""
from __future__ import annotations
import os
from typing import Any
from pydantic import BaseModel, Field
_MAX_BATCH = int(os.getenv("APPLOG_MAX_BATCH", "500"))
class AppLogBatchIn(BaseModel):
device_id: str = Field(max_length=64)
user_id: int | None = None
app_ver: str | None = Field(default=None, max_length=32)
platform: str | None = Field(default=None, max_length=16)
sent_at: int | None = None
logs: list[dict[str, Any]] = Field(min_length=1, max_length=_MAX_BATCH)
class AppLogIngestOut(BaseModel):
ok: bool = True
received: int
dropped: int = 0
```
- [ ] **Step 4: 实现端点 `app/api/v1/applog.py`**
```python
"""客户端运行日志批量上报接口。
POST /api/v1/applog/batch — 批量接收客户端运行日志,逐条写专用滚动文件 logs/app-client.log
(供 Logtail 采进独立 SLS logstore)。鉴权同 analytics(不强制登录,user_id 可选在 body)。
fire-and-forget:写失败也不 500(避免客户端重试风暴);超批 422、超体积 413、msg 超限截断。
"""
from __future__ import annotations
import os
from fastapi import APIRouter, Depends, HTTPException, Request
from app.core.client_log import write_records
from app.core.ratelimit import rate_limit
from app.schemas.applog import AppLogBatchIn, AppLogIngestOut
router = APIRouter(prefix="/api/v1/applog", tags=["applog"])
def _client_ip(request: Request) -> str:
"""取客户端 IP:生产经 nginx 反代优先 X-Forwarded-For 首段,否则直连 IP(同 analytics)。"""
xff = request.headers.get("x-forwarded-for")
if xff:
return xff.split(",")[0].strip()
return request.client.host if request.client else ""
def _enforce_body_limit(request: Request) -> None:
"""依赖:body 声明过大直接 413(在 body 校验前拦截)。缺 Content-Length 由 nginx 兜底。"""
max_bytes = int(os.getenv("APPLOG_MAX_BODY_BYTES", str(1024 * 1024 * 2)))
cl = request.headers.get("content-length")
if cl is not None and cl.isdigit() and int(cl) > max_bytes:
raise HTTPException(status_code=413, detail="日志批量过大")
@router.post(
"/batch",
response_model=AppLogIngestOut,
summary="批量上报客户端运行日志",
dependencies=[
Depends(rate_limit(120, 60, "applog-batch")),
Depends(_enforce_body_limit),
],
)
def ingest_logs(batch: AppLogBatchIn, request: Request) -> AppLogIngestOut:
received, dropped = write_records(
batch.logs,
meta={
"device_id": batch.device_id,
"user_id": batch.user_id,
"app_ver": batch.app_ver,
"platform": batch.platform,
"sent_at": batch.sent_at,
},
client_ip=_client_ip(request),
)
return AppLogIngestOut(received=received, dropped=dropped)
```
- [ ] **Step 5: 注册路由 `app/main.py`**
在 import 区(analytics_router 之后,约 [app/main.py:23](../../../app/main.py#L23))加:
```python
from app.api.v1.applog import router as applog_router
```
`include_router` 区(`app.include_router(analytics_router)` 之后,约 [app/main.py:125](../../../app/main.py#L125))加:
```python
app.include_router(applog_router)
```
- [ ] **Step 6: 跑测试确认通过**
Run: `pytest tests/test_applog.py -q`
Expected: PASSwriter 5 + 端点 5,共 10 个)。
- [ ] **Step 7: 全量测试 + ruff**
Run: `pytest -q && ruff check app/api/v1/applog.py app/schemas/applog.py app/main.py`
Expected: 全绿、无 lint 错误。
- [ ] **Step 8: 提交**
```bash
git add app/api/v1/applog.py app/schemas/applog.py app/main.py tests/test_applog.py
git commit -m "feat(applog): POST /api/v1/applog/batch 批量上报端点(限流+体积上限+fire-and-forget)"
```
---
## Task 3: `.env.example` 文档化新 env(可选但推荐)
**Files:**
- Modify: `.env.example`
- [ ] **Step 1: 追加 env 说明**
`.env.example` 末尾(或日志相关区块)追加,让运维知道可调项:
```bash
# 客户端运行日志上报(POST /api/v1/applog/batch)→ 落 logs/app-client.log 供 Logtail 采集
# 独立于服务日志 app-server.log;滚动机制同服务日志。默认值见 app/core/client_log.py。
# CLIENT_LOG_FILE=logs/app-client.log
# CLIENT_LOG_MAX_BYTES=20971520 # 单文件 20MB 滚动
# CLIENT_LOG_BACKUP_COUNT=10 # 保留 10 个 → ~200MB 缓冲
# CLIENT_LOG_SERVICE_NAME=app-client # 输出行 service 字段
# APPLOG_MAX_BATCH=500 # 单批最大条数(超 → 422)
# APPLOG_MAX_BODY_BYTES=2097152 # 请求体上限 2MB(超 → 413)
# APPLOG_MAX_MSG_BYTES=8192 # 单条 msg 超限截断
```
- [ ] **Step 2: 提交**
```bash
git add .env.example
git commit -m "docs(applog): .env.example 补充客户端日志上报可调 env"
```
> 若仓库无 `.env.example`(以 `git ls-files .env.example` 确认),跳过本任务。
---
## Definition of Done
- [ ] `pytest -q` 全绿(含新增 `tests/test_applog.py` 10 用例)。
- [ ] `ruff check .` 无错误。
- [ ] `POST /api/v1/applog/batch` 手动冒烟:发一批含 `trace_id` 的日志,确认 `logs/app-client.log` 出现对应单行 JSON、`trace_id` 在顶层、未知键落在 `data`,且 `logs/app-server.log` **未**被写入客户端记录。
- [ ] Spec 的运维项(独立 logstore、Logtail JSON 模式采 `app-client.log`、给 `trace_id` 建索引、nginx `client_max_body_size` 对齐)已同步给运维(不在本仓代码内,见 spec §8)。
## Self-Review 结论(作者已核对)
- **Spec 覆盖**:§4 端点/限额 → Task 2;§5 白名单+data 兜底 → Task 1 `_build_line`;§6 落盘/滚动/单 worker 注释 → Task 1;§7 trace_id 顶层 → Task 1 + 端点测试;§9 fire-and-forget → Task 1 `write_records` + 端点测试;§13 env → Task 1/2 读取 + Task 3 文档化。§8 为纯运维配置,列入 Definition of Done。
- **无占位符**:所有步骤含完整代码/命令/预期。
- **类型/命名一致**`write_records(records, *, meta, client_ip) -> (received, dropped)``get_logger()``reset_client_logger()``_TOP_LEVEL_KEYS``AppLogBatchIn/AppLogIngestOut` 在 Task 1/2 间一致引用。
@@ -0,0 +1,323 @@
# 允许在途提现时继续提交新申请 Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** 取消「同一用户同一时刻仅一笔在途提现」限制,已有 `reviewing`/`pending` 单时允许继续提交新的提现申请。
**Architecture:** 该限制由两道闸共同强制——应用层 `create_withdraw` 的在途单检查(`WithdrawTooFrequentError`)与数据库分区唯一索引 `ux_withdraw_order_user_active`。彻底移除两者 + 清理随之失效的死代码;既有约束(建单先扣款、`coin_cash` 每日档位次数、`out_bill_no` 幂等)天然保留,无需改动。测试库由 `Base.metadata.create_all()` 依模型建表,故删模型内索引定义即让测试反映新 schema;另配一条 Alembic 迁移让真实库(dev/prod)落地同一变更。
**Tech Stack:** FastAPI · SQLAlchemy 2.0 · Alembic · pytest · ruff
规格来源:[docs/superpowers/specs/2026-07-24-withdraw-allow-concurrent-design.md](../specs/2026-07-24-withdraw-allow-concurrent-design.md)
---
## 关键事实(实现依据)
- 当前 Alembic head:`d8dd2106e438`(新迁移的 `down_revision`)。
- 索引出处:[`alembic/versions/withdraw_safety_indexes.py`](../../../alembic/versions/withdraw_safety_indexes.py) 同时建了两个索引——本次**只删** `ux_withdraw_order_user_active`,**保留**姊妹索引 `ux_cash_transaction_withdraw_refund_ref`(退款幂等)。
- 档位:`WithdrawTier(50, "0.5", None, 3, False)`——50 分(0.5 元)是**常规档**,每日 3 次,非新人档;测试用它来造多笔在途。
- 测试库走 `create_all`(见 `tests/conftest.py`),**不跑 Alembic**;故迁移的正确性由本计划单独的 upgrade/downgrade 回环验证,不由 pytest 覆盖。
- `app/models/wallet.py``Index`/`text` 仍被其它表(行 54/186/224)使用,删本表 `__table_args__` 后**无需**清理 import。
---
## Task 1: 允许多笔在途提现并存(TDD:模型 + 应用层 + 端点 + 迁移,单次提交)
本变更是一次原子的 schema+行为改动:模型内索引、应用层检查、Alembic 迁移相互依赖,任一缺失都会让"多笔在途"在测试库或真实库其一不成立。故作为**一个任务、一次提交**完成,内部按 TDD 分步。
**Files:**
- Test: `tests/test_withdraw.py`(新增 2 个用例)
- Modify: `app/repositories/wallet.py`(删在途单检查 + IntegrityError 兜底瘦身 + 删死常量/异常)
- Modify: `app/api/v1/wallet.py:225-229`(删失效的 409 处理)
- Modify: `app/models/wallet.py:99-107`(删分区唯一索引)
- Create: `alembic/versions/drop_withdraw_active_unique_index.py`(真实库删索引)
---
- [ ] **Step 1: 写两个失败测试**
`tests/test_withdraw.py` 末尾追加(复用文件内既有 helper `_login`/`_auth`/`_seed_cash`/`_patch_userinfo`):
```python
def test_withdraw_multiple_in_flight_allowed(client, monkeypatch) -> None:
"""取消「同时仅一单」:已有在途(reviewing)时,不同 out_bill_no 可继续提交,两单并存。"""
_patch_userinfo(monkeypatch, "openid_multi_inflight")
token = _login(client, "13800002020")
_seed_cash(client, token, "13800002020", 100) # 够两笔 0.5
client.post("/api/v1/wallet/bind-wechat", json={"code": "c"}, headers=_auth(token))
r1 = client.post(
"/api/v1/wallet/withdraw",
json={"amount_cents": 50, "out_bill_no": "billmulti00000001"},
headers=_auth(token),
)
assert r1.status_code == 200, r1.text
assert r1.json()["status"] == "reviewing"
r2 = client.post(
"/api/v1/wallet/withdraw",
json={"amount_cents": 50, "out_bill_no": "billmulti00000002"},
headers=_auth(token),
)
assert r2.status_code == 200, r2.text
assert r2.json()["status"] == "reviewing"
# 两张在途单并存
r = client.get("/api/v1/wallet/withdraw-orders", headers=_auth(token))
reviewing = [o for o in r.json()["items"] if o["status"] == "reviewing"]
assert len(reviewing) == 2, r.text
# 余额扣两次:100-50-50=0
r = client.get("/api/v1/wallet/account", headers=_auth(token))
assert r.json()["cash_balance_cents"] == 0
def test_withdraw_second_blocked_only_by_insufficient_cash(client, monkeypatch) -> None:
"""并行放开后第二笔仅受余额约束:余额不足返 409「现金余额不足」,而非旧的「已有提现」拦截。"""
_patch_userinfo(monkeypatch, "openid_multi_insuff")
token = _login(client, "13800002021")
_seed_cash(client, token, "13800002021", 50) # 仅够一笔 0.5
client.post("/api/v1/wallet/bind-wechat", json={"code": "c"}, headers=_auth(token))
r1 = client.post(
"/api/v1/wallet/withdraw",
json={"amount_cents": 50, "out_bill_no": "billinsuff0000001"},
headers=_auth(token),
)
assert r1.status_code == 200, r1.text
r2 = client.post(
"/api/v1/wallet/withdraw",
json={"amount_cents": 50, "out_bill_no": "billinsuff0000002"},
headers=_auth(token),
)
assert r2.status_code == 409, r2.text
assert "现金余额不足" in r2.json()["detail"]
```
- [ ] **Step 2: 运行新测试,确认失败**
Run: `pytest tests/test_withdraw.py::test_withdraw_multiple_in_flight_allowed tests/test_withdraw.py::test_withdraw_second_blocked_only_by_insufficient_cash -q`
Expected: 两条 FAIL —— `multiple_in_flight` 因第二笔被拦返回 409(期望 200);`second_blocked` 因返回的 409 detail 是「已有提现申请正在审核或打款中」而非「现金余额不足」。
- [ ] **Step 3: 应用层删在途单互斥检查**(`app/repositories/wallet.py` · `create_withdraw`)
删掉在途单预检查(它紧邻档位闸注释之前):
old:
```python
active_order_id = db.execute(
select(WithdrawOrder.id).where(
WithdrawOrder.user_id == user_id,
WithdrawOrder.status.in_(_WITHDRAW_ACTIVE_STATUSES),
).limit(1)
).scalar_one_or_none()
if active_order_id is not None:
raise WithdrawTooFrequentError
# 福利页档位闸(7-9):coin_cash 只能提预设档位,且该档今日可提(服务端权威口径,防绕过
```
new:
```python
# 福利页档位闸(7-9):coin_cash 只能提预设档位,且该档今日可提(服务端权威口径,防绕过
```
- [ ] **Step 4: 应用层给 IntegrityError 兜底瘦身**(同函数末尾 commit 处)
索引移除后不会再因在途单触发唯一冲突,只保留 `out_bill_no` 幂等重试分支:
old:
```python
except IntegrityError:
db.rollback()
existing = db.execute(
select(WithdrawOrder).where(
WithdrawOrder.out_bill_no == out_bill_no, WithdrawOrder.user_id == user_id
)
).scalar_one_or_none()
if existing is not None:
return existing
active_order_id = db.execute(
select(WithdrawOrder.id).where(
WithdrawOrder.user_id == user_id,
WithdrawOrder.status.in_(_WITHDRAW_ACTIVE_STATUSES),
).limit(1)
).scalar_one_or_none()
if active_order_id is not None:
raise WithdrawTooFrequentError from None
raise
```
new:
```python
except IntegrityError:
db.rollback()
# 唯一冲突只可能来自 out_bill_no 幂等键并发重试:原样返回既有单;否则未知冲突,上抛。
existing = db.execute(
select(WithdrawOrder).where(
WithdrawOrder.out_bill_no == out_bill_no, WithdrawOrder.user_id == user_id
)
).scalar_one_or_none()
if existing is not None:
return existing
raise
```
- [ ] **Step 5: 删死常量 `_WITHDRAW_ACTIVE_STATUSES`**(`app/repositories/wallet.py` 模块顶部)
只删常量行,保留其后属于 `_NEWBIE_TIER_HELD_STATUSES` 的注释:
old:
```python
_WITHDRAW_ACTIVE_STATUSES = {"reviewing", "pending"}
# 占用新人档「一次性」资格的提现状态:进行中(reviewing/pending)或成功打款(success)。
```
new:
```python
# 占用新人档「一次性」资格的提现状态:进行中(reviewing/pending)或成功打款(success)。
```
- [ ] **Step 6: 删死异常类 `WithdrawTooFrequentError`**(`app/repositories/wallet.py`)
old:
```python
class WithdrawTooFrequentError(Exception):
"""提现申请过于频繁,或已有未完成提现单。"""
class WithdrawTierUnavailableError(Exception):
```
new:
```python
class WithdrawTierUnavailableError(Exception):
```
- [ ] **Step 7: 删端点内失效的 409 处理**(`app/api/v1/wallet.py` · `withdraw`)
old:
```python
except crud_wallet.WechatNotBoundError as e:
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="请先绑定微信") from e
except crud_wallet.WithdrawTooFrequentError as e:
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail="已有提现申请正在审核或打款中,请处理完成后再申请",
) from e
except crud_wallet.WithdrawTierUnavailableError as e:
```
new:
```python
except crud_wallet.WechatNotBoundError as e:
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="请先绑定微信") from e
except crud_wallet.WithdrawTierUnavailableError as e:
```
- [ ] **Step 8: 删模型内分区唯一索引**(`app/models/wallet.py` · `WithdrawOrder`)
old:
```python
__tablename__ = "withdraw_order"
__table_args__ = (
Index(
"ux_withdraw_order_user_active",
"user_id",
unique=True,
sqlite_where=text("status IN ('reviewing', 'pending')"),
postgresql_where=text("status IN ('reviewing', 'pending')"),
),
)
```
new:
```python
__tablename__ = "withdraw_order"
```
- [ ] **Step 9: 运行新测试,确认通过**
Run: `pytest tests/test_withdraw.py::test_withdraw_multiple_in_flight_allowed tests/test_withdraw.py::test_withdraw_second_blocked_only_by_insufficient_cash -q`
Expected: 2 passed。
- [ ] **Step 10: 建 Alembic 迁移(真实库删索引)**
创建 `alembic/versions/drop_withdraw_active_unique_index.py`:
```python
"""drop withdraw active-order partial unique index (allow multiple in-flight withdrawals)
Revision ID: drop_withdraw_active_unique_index
Revises: d8dd2106e438
Create Date: 2026-07-24 00:00:00.000000
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = "drop_withdraw_active_unique_index"
down_revision: Union[str, Sequence[str], None] = "d8dd2106e438"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# 取消「同一用户同一时刻仅一笔在途提现」:允许 reviewing/pending 并存。
# 仅删本索引;姊妹索引 ux_cash_transaction_withdraw_refund_ref(退款幂等)保持不动。
op.drop_index("ux_withdraw_order_user_active", table_name="withdraw_order")
def downgrade() -> None:
# 回滚重建分区唯一索引。注意:若届时某用户已有 ≥2 张在途单,重建会因唯一冲突失败——
# 属预期的回滚代价(取消限制后本就允许多单),需先人工收敛在途单再回滚。
op.create_index(
"ux_withdraw_order_user_active",
"withdraw_order",
["user_id"],
unique=True,
sqlite_where=sa.text("status IN ('reviewing', 'pending')"),
postgresql_where=sa.text("status IN ('reviewing', 'pending')"),
)
```
- [ ] **Step 11: 验证迁移 upgrade + 回环 downgrade/upgrade**
Run: `alembic upgrade head`
Expected: 输出应用 `drop_withdraw_active_unique_index`,无报错。
Run: `alembic downgrade -1 && alembic upgrade head`
Expected: downgrade 重建索引、upgrade 再次删除,均成功(dev 库每用户在途单 ≤1,不会触发唯一冲突)。
- [ ] **Step 12: 全量测试 + Lint**
Run: `pytest -q`
Expected: 全绿(既有 `test_withdraw_idempotent_same_bill_no``tests/test_withdraw_tiers.py``tests/test_invite_cash_withdraw.py` 均不受影响)。
Run: `ruff check .`
Expected: 无新增告警(死常量/异常已连同引用一并删除)。
- [ ] **Step 13: 提交**
```bash
git add tests/test_withdraw.py app/repositories/wallet.py app/api/v1/wallet.py app/models/wallet.py alembic/versions/drop_withdraw_active_unique_index.py
git commit -m "feat(withdraw): 允许在途提现时继续提交新申请
取消「同一用户同时仅一笔在途提现」限制:删应用层在途单检查 +
删 DB 分区唯一索引 ux_withdraw_order_user_active + 清理死代码
(_WITHDRAW_ACTIVE_STATUSES / WithdrawTooFrequentError)。
先扣款、coin_cash 每日档位次数、out_bill_no 幂等等既有约束不变。
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>"
```
---
## 自审(spec 覆盖核对)
- **R1 已有 reviewing/pending 可再提** → Step 3/4(应用层)+ Step 8/10(模型 + 迁移);`test_withdraw_multiple_in_flight_allowed` 证明。✓
- **R2 不加硬上限** → 无新增限制;`coin_cash` 天然封顶由既有档位次数(`tests/test_withdraw_tiers.py`)保障,不改。✓
- **R3 幂等/限额/退款/对账不回退**`test_withdraw_idempotent_same_bill_no` 与档位/邀请现金测试留绿(Step 12);解绑退款、对账按单号维度,未触碰。✓
- **spec §4 四处改动** → Step 3/4/5/6(repo)、Step 7(api)、Step 8(model)、Step 10(迁移)一一对应。✓
- **spec §6 客户端注意点** → 属跨仓 App 事项,文档已记,本计划无对应代码任务(有意为之)。✓
- **占位符扫描**:迁移 `revision` / `down_revision`(`d8dd2106e438`)均为具体值,无 TBD。✓
- **命名一致性**:`WithdrawTooFrequentError`/`_WITHDRAW_ACTIVE_STATUSES` 的删除点与引用点全覆盖(全仓仅这 4 处);`ux_withdraw_order_user_active` 在模型/迁移中拼写一致。✓
@@ -0,0 +1,200 @@
# 客户端运行日志批量上报 → 落专用文件 → SLS 采集 设计
- 日期:2026-07-19
- 状态:设计已评审,待写实现计划
- 相关代码:[app/core/logging.py](../../../app/core/logging.py)(服务端日志落盘范式)、[app/api/v1/analytics.py](../../../app/api/v1/analytics.py)(批量上报入口范式)、[deploy/shaguabijia-app-server.service](../../../deploy/shaguabijia-app-server.service)`--workers 1`
## 1. 背景与目标
Android 客户端会在本地攒一批 App 运行日志(自动化步骤 / 网络 / 崩溃 / 调试等),批量以 JSON 数组发给后端。后端把这些日志**逐条**写进一个**专用滚动日志文件**,文件滚动规则与服务日志一致;再由阿里云 SLS 的 Logtail 采集该文件、进独立 logstore。
目标:
1. 新增 `POST /api/v1/applog/batch` 接收批量客户端日志。
2. 逐条落进 `logs/app-client.log`**独立于** `app-server.log`),单行 JSON、大小滚动、供 Logtail JSON 模式零正则采集。
3. 客户端日志进**独立 logstore**,但 **可按 `trace_id` 检索**,以便和服务端比价链路跨层对齐。
4. 入口安全:鉴权同 analytics(不强制 Bearer),加体积/条数上限、限流,防撑盘与日志注入。
本方案是两个**已验证范式的组合**:analytics 的「批量入口 + 服务端补 IP/接收时间」+ `app-server.log` 的「单行 JSON + RotatingFileHandler + Logtail」。不是新架构。
## 2. 非目标(本期不做,YAGNI)
- **异步队列 / 后台 writer**`<10 万条/天` ≈ 均值 1.2 条/秒,10× 突发 ~12/秒,请求路径同步写文件足矣。
- **去重**:网络重试会在 SLS 产生重复条目。本期**接受重复**并在文档写明;后续如需,加客户端 `batch_id` 短窗口去重。
- **服务端脱敏**:本期靠**客户端侧**控制等级/采样与不打 PII;服务端只做体积截断。脱敏留作后续 knob。
- **多 worker 支持**:见 §5 的单 worker 约束。
- **入库 / admin 查询界面**:日志的归宿是 SLS,不落 DB。
## 3. 方案总览与备选取舍
**采用 A**:客户端 → 后端接口 → 专用滚动文件 → Logtail → SLS。
| 方案 | 说明 | 为何不选 |
|---|---|---|
| **A(选定)** | 后端中转落文件,Logtail 采集 | — 代码最少、完全复用现成文件→Logtail 管线;AK/SK 不进 APKLogtail 天然提供落盘缓冲+断点续传 |
| B | 客户端直连 SLSProducer SDK / Web-Tracking | 要么把凭证埋进 APK,要么额外跑 STS 换 token 服务;服务端难做鉴权/富化/脱敏 |
| C | 后端调 SLS PutLogs API(不落文件) | 请求路径硬依赖 SLS 可用性,需自建缓冲/背压/重试——等于重造 Logtail |
## 4. 端点契约
`POST /api/v1/applog/batch`
- 新路由 `app/api/v1/applog.py`,在 [app/main.py](../../../app/main.py) `import ... as applog_router``app.include_router(applog_router)`(紧挨 analytics)。
- 鉴权同 analytics**不强制 Bearer**`user_id` 可选放 body;服务端补 `client_ip`(复用 analytics 里的 `_client_ip``X-Forwarded-For` 首段逻辑)与接收时间。
### 请求体 `AppLogBatchIn`(批级公共字段发一次,省带宽)
| 字段 | 类型 | 必填 | 约束/说明 |
|---|---|---|---|
| `device_id` | str | 是 | `max_length=64`;限流/分组键 |
| `user_id` | int? | 否 | 可选,未登录态也采集 |
| `app_ver` | str? | 否 | `max_length=32` |
| `platform` | str? | 否 | `max_length=16`android/ios/harmony |
| `sent_at` | int? | 否 | 本批上报时刻 epoch ms |
| `logs` | list[dict] | 是 | `min_length=1, max_length=500`;**每条为对象**,逐条按 §5 契约处理 |
`logs``list[dict[str, Any]]` 而非强类型列表:这是**尽力而为**的日志链路,单条内容异常不应让整批 422 失败。超过 500 条由 Pydantic `max_length` 触发 422(客户端应更小批)。
### 响应 `AppLogIngestOut`
```jsonc
{ "ok": true, "received": 128, "dropped": 2 }
```
- `received`:成功写入文件的条数。
- `dropped`:服务端处理/序列化失败被跳过的条数(正常为 0,属异常兜底计数)。**oversize 的 `msg` 是截断而非丢弃**,不计入 dropped。
### 限额(防撑盘 / 注入 / DoS)
| 限额 | 默认 | 超限行为 | env |
|---|---|---|---|
| 单批条数 | 500 | 422Pydantic | `APPLOG_MAX_BATCH` |
| body 字节 | 2 MB | 413(依赖查 `Content-Length`,在 body 校验前拦截;缺该头由 nginx `client_max_body_size` 兜底) | `APPLOG_MAX_BODY_BYTES` |
| 单条 `msg` 字节 | 8192 | 截断 + 标记,不丢 | `APPLOG_MAX_MSG_BYTES` |
同时**对齐 nginx `client_max_body_size`**(见 [deploy/nginx](../../../deploy/nginx/))避免反代先于应用截断。限流复用 `app/core/ratelimit.py`(项目现有 IP 固定窗口);具体挂法参照现有已限流写端点,测试环境 `RATE_LIMIT_ENABLED=false` 关闭。
## 5. 每条记录契约 + 白名单键 + `data` 兜底
**客户端每条日志的识别键(仅这些提到输出行顶层):**
| 键 | 类型 | 说明 |
|---|---|---|
| `client_ts` | int | 端事件时间 epoch ms(与 analytics 命名一致) |
| `level` | str | 服务端归一化为大写;不做硬枚举拒绝(fire-and-forget |
| `trace_id` | str? | **§6 的核心**:有服务端交互的日志带上当初 API 返回的 trace |
| `tag` | str? | 模块/分类,便于 SLS 过滤 |
| `msg` | str | 消息主体,超 `APPLOG_MAX_MSG_BYTES` 截断并加标记 |
**其余任意自定义字段 → 服务端一律收进单个 `data` 对象。**
**为什么这么设计(防 SLS 索引列爆炸):** Logtail JSON 模式下每个**顶层 key** 都会成为 logstore 一个可索引字段/列。若放任客户端往顶层写任意 key(甚至把动态 id 拼进 key),logstore 会长出成千上万个不同顶层列 → 索引成本膨胀、可能撞字段数上限、schema 混乱到无法建稳定 dashboard/告警,一个客户端 bug 就能把 logstore 搞脏。
**「兜底」= 服务端强制、不信任客户端。** `logs``list[dict]` 原样收下(不强类型,单条异常不该让整批 422),在**写入层 `client_log.py`** 逐条拆分:白名单键提顶层,**其余键(含客户端自带的 `data`)统一并进 `data`**(而非透传到顶层,也非静默丢弃——日志要保真)。于是**输出行顶层列恒定**,无论客户端怎么发。
这与现有 `JsonFormatter` 平铺 `phase/step/command/cost_ms` 同思路,区别:那些 extra 是**服务端可信有限**的键;客户端不可信无界,故只给一个 `data` 沙盒。
## 6. 落盘 writer
新模块 `app/core/client_log.py`
- 惰性单例 `logging.getLogger("client")`**`propagate=False`**(否则冒泡进 root 被 `app-server.log` 二次写入并污染)。
- 独占一个 `RotatingFileHandler`formatter 为 `%(message)s`——**不复用 root 的 `JsonFormatter`**(那会把已是 JSON 的行二次编码成字符串)。writer 自己拼信封 dict 后 `json.dumps(..., ensure_ascii=False, default=str)` 得到**一行**`logger.info(line)` 写出。
- 复用 `logging` 模块的 handler 锁保证多线程(uvicorn threadpool)并发写安全。
- 幂等 setup(仿 `setup_logging``_CONFIGURED` 守卫)。
**滚动规则(与服务日志同机制 `RotatingFileHandler`,尺寸给客户端量级):**
| 参数 | 默认 | env |
|---|---|---|
| 文件路径 | `logs/app-client.log` | `CLIENT_LOG_FILE` / `LOG_DIR` |
| `maxBytes` | 20 MB | `CLIENT_LOG_MAX_BYTES` |
| `backupCount` | 10 | `CLIENT_LOG_BACKUP_COUNT` |
| `service` 字段 | `app-client` | `CLIENT_LOG_SERVICE_NAME` |
≈ 200 MB / ~2 天缓冲,给 Logtail 断线留余量(按 <10 万条/天、条均值估算)。**滚动文件不 gzip**——Logtail 读不了压缩包会丢数据。
> ⚠️ **单 worker 约束**`RotatingFileHandler` 多进程并发 `doRollover()` 会损坏/丢日志。当前生产 `--workers 1`(与限流器/SMS 码/SQLite 写锁同一既有假设,见 [deploy/shaguabijia-app-server.service](../../../deploy/shaguabijia-app-server.service))故安全。**代码注释显式标注**:扩 worker 前必须换 `QueueHandler`→单写入者 或外部 logrotatecopytruncate)或写 stdout 交 journald 采集。
**每条输出行(写入 `app-client.log`):**
```jsonc
{ "time": "2026-07-19T12:00:00.123", // 服务端接收时间 = SLS 主时间(客户端时钟不可信)
"source": "client", "service": "app-client",
"level": "ERROR", "trace_id": "abc123", "tag": "automation", "msg": "...",
"device_id": "d-xxx", "user_id": 123, "app_ver": "1.2.3", "platform": "android",
"client_ip": "1.2.3.4", "client_ts": 1737000000123, "sent_at": 1737000005000,
"data": { /* 客户端其余任意字段 */ } }
```
- `time` 用服务端接收时间作 SLS 主时间;`client_ts` 保留为可查字段(时钟漂移不影响检索基准)。
- 值为空的可选字段省略,保持行精简(仿服务端 formatter 省略空 `trace_id`)。
## 7. trace_id 检索(满足目标 #3
关键:**per-record 把 `trace_id` 提到输出行顶层,字段名与服务端日志完全一致(`trace_id`)**。来源是客户端在**它发起过服务端调用的那些日志**里带上当初 API 返回的 trace(如 `/api/v1/compare/*` 由 [pricebot_router](../../../app/core/pricebot_router.py) 透传的 trace)。没有服务端交互的纯客户端日志不带 `trace_id`,正常。
于是 SLS 里对客户端 logstore 与服务端 logstore 各查 `trace_id: "xxx"`,即可拼出「客户端自动化视角 + 服务端比价链路」的端到端故事。
> **需客户端配合**:给有服务端 trace 的日志记录打上 `trace_id`(Android 侧改动,见 §10)。
## 8. Logtail / SLS 侧配置(运维,非本仓代码)
- 新建**独立 logstore**(独立保留期,客户端日志建议**比服务端短**以控成本)。
- 新 Logtail 配置采集 `logs/app-client.log`**JSON 模式**(每行一条 JSON,零正则,与 `app-server.log` 同套路)。
- **把 `trace_id` 配成索引字段**(否则 #3 查不了);`level` / `tag` / `device_id` 亦建议建索引。
- 时间字段用输出行 `time`(服务端接收时间)。
## 9. 失败语义 / 安全
- **fire-and-forget**:写文件失败 → 服务端记一笔(`shagua.applog` logger+ 仍返回 2xx**绝不 500**(对比 selfstat 的 503 是 DB 关键链路,日志不是;500 会招致客户端重试风暴+重复上报)。
- **日志行注入防护**:一律 `json.dumps` 重新序列化,内嵌 `\n` 被转义,客户端伪造不出假日志行;绝不把客户端原始字符串直接写文件。
- **撑盘/DoS**:§4 的条数/体积/msg 上限 + IP 限流;`device_id` 便于后续拉黑滥用设备。
## 10. 客户端契约(Android 侧需配合,属另一仓)
1. 每条日志结构:顶层放 `client_ts / level / trace_id? / tag? / msg`,其余自定义字段放 `data`(否则会被服务端兜底挪进 `data`)。
2. 有服务端交互的日志带上对应 `trace_id`
3. 等级/采样与 PII 控制在客户端侧做(省流量、免服务端脱敏)。
4. 单批 ≤500 条、body ≤2MB;失败可重试(服务端接受重复)。
## 11. 代码落点
| 文件 | 职责 |
|---|---|
| `app/api/v1/applog.py` | 瘦路由:body 上限依赖 → 解析 `AppLogBatchIn` → 调 writer → 返回 `received/dropped``_client_ip` 复用 analytics 逻辑、IP 限流 |
| `app/schemas/applog.py` | `AppLogBatchIn``logs: list[dict]` 不强类型)/ `AppLogIngestOut` |
| `app/core/client_log.py` | 专用 logger/handler、滚动配置、白名单+`data` 兜底与信封拼装 `write_records(...)` |
| `app/main.py` | 注册 `applog_router` |
## 12. 测试计划(`tests/test_applog.py`
用临时目录做 `CLIENT_LOG_FILE`(测试前置 env + 重置 writer 单例)。
1. **落盘逐行**POST N 条 → 200`received=N`,文件恰 N 行、每行合法 JSON、关键字段齐。
2. **trace_id 顶层**:带 `trace_id` 的记录 → 输出行顶层出现 `trace_id`
3. **未知键兜底**:记录带 `foo` → 输出行顶层无 `foo``data.foo` 存在。
4. **msg 截断**`msg` > 8KB → 截断+标记,仍 `received`(不进 dropped)。
5. **超批拒绝**>500 条 → 422。
6. **超体积拒绝**`Content-Length` > 2MB → 413。
7. **写失败不 500**monkeypatch writer 抛错 → 仍 2xx。
8. **不污染服务日志**`client` logger `propagate=False`,写客户端日志不落 `app-server.log`
## 13. env 变量汇总
| env | 默认 | 用途 |
|---|---|---|
| `CLIENT_LOG_FILE` | `logs/app-client.log` | 客户端日志文件路径 |
| `CLIENT_LOG_MAX_BYTES` | `20971520`(20MB) | 单文件滚动阈值 |
| `CLIENT_LOG_BACKUP_COUNT` | `10` | 保留滚动文件数 |
| `CLIENT_LOG_SERVICE_NAME` | `app-client` | 输出行 `service` 字段 |
| `APPLOG_MAX_BATCH` | `500` | 单批最大条数 |
| `APPLOG_MAX_BODY_BYTES` | `2097152`(2MB) | 请求体上限 |
| `APPLOG_MAX_MSG_BYTES` | `8192` | 单条 msg 截断阈值 |
滚动/文件类 env 在 `client_log.py``os.getenv` 读取(与 [logging.py](../../../app/core/logging.py) 风格一致);请求限额类同样以 `os.getenv` 兜默认。
## 14. 已知取舍 / 未来工作
- **重复**:本期接受 SLS 重复条目;需要时加 `batch_id` 去重。
- **多 worker**:见 §6 约束;扩容前迁移写入模型。
- **服务端脱敏**:留作后续 knob。
- **更多设备维度**os/model/rom):需要时加到批级字段或让客户端放 `data`;可经 `device_id` 与 analytics/device 表关联。
@@ -0,0 +1,119 @@
# 允许在途提现时继续提交新的提现申请 设计
- **日期**:2026-07-24
- **状态**:Draft — 待评审
- **所属**:app-server(`app/`),含一处 Alembic 迁移;另有一条 Android/客户端注意点(非本次后端工作)
- **一句话**:取消「同一用户同一时刻只能有一笔在途提现」的限制 —— 已有 `reviewing`(待审核)或 `pending`(打款在途)提现单时,允许再次发起新的提现申请。
---
## 1. 背景与目标
现状:用户发起提现后,在管理员审核通过并打款完成之前(`reviewing` / `pending`),**无法再发起第二笔提现**,会收到 409「已有提现申请正在审核或打款中,请处理完成后再申请」。人工审核有延迟时,用户被卡住、体验差。
**目标**:放开该限制,已有在途提现单时仍可继续提交新的提现申请。
### 非目标(本期不做)
- 不改「先扣款」模型(提现建单即原子扣现金,天然防超提)。
- 不改 `coin_cash` 的每日档位次数限制(仍是独立的限额闸)。
- 不加任何新的并发/次数硬上限(见 §3 决策)。
- 不改 admin 审核/打款/对账逻辑(其全部按单号维度操作,天然支持一人多单)。
---
## 2. 需求
| # | 需求 | 落地 |
|---|---|---|
| R1 | 已有 `reviewing``pending` 单时可再次发起提现 | 删除应用层在途单互斥检查 + 删除 DB 分区唯一索引(§4) |
| R2 | 不引入新的并发上限 | 仅靠既有约束:先扣款余额 + `coin_cash` 每日档位次数(§5) |
| R3 | 既有幂等/限额/退款/对账行为不回退 | 保留 `out_bill_no` 幂等、档位闸、解绑退款、对账(§5) |
---
## 3. 决策记录(来自评审问答)
| 决策点 | 结论 | 理由 |
|---|---|---|
| **放行范围** | `reviewing``pending` **两种在途状态都放行**,彻底取消「同时仅一单」 | 需求即"存在在途提现时可继续提交";人工审核延迟不应卡住用户 |
| **并发上限** | **不加硬上限** | 现金**先扣款**→每笔各需自己的余额,不会超提;`coin_cash` 每日档位次数已是天然上限;`invite_cash` 仅受余额约束,可接受 |
| **实现方式** | **彻底移除**(删检查 + 删索引 + 清理死代码),非配置开关 | 决策明确且无回滚诉求,YAGNI;避免留半死代码与多余配置项 |
| **已失效异常/文案** | 删除 `WithdrawTooFrequentError` 及端点的 409 处理 | 全仓仅这 4 处引用(§4),移除后无残留 |
### 已知取舍(可接受)
- **多笔 `pending` × 未开免确认**:每笔 `pending` 会各自返回一个微信确认页 `package_info`。用户若**未开启免确认**,可能同时存在多笔待确认。属客户端交互问题(§6),后端行为正确;开启免确认后直接到账、无此问题。
- **回滚风险**:迁移 `downgrade` 会重建分区唯一索引;若届时某用户已有 ≥2 张在途单,重建会失败 —— 属预期的回滚代价,在迁移里注释说明。
---
## 4. 现状机制与改动点
「同一时刻仅一笔在途提现」由**两道闸**共同强制,均以 `status IN ('reviewing','pending')` 为口径:
1. **应用层** — [`app/repositories/wallet.py:758-765`](../../../app/repositories/wallet.py) `create_withdraw` 内:查在途单 → `raise WithdrawTooFrequentError` → 端点转 409。
2. **数据库层** — [`app/models/wallet.py:99-107`](../../../app/models/wallet.py) 的**分区唯一索引** `ux_withdraw_order_user_active`(`user_id WHERE status IN ('reviewing','pending')`)。这是硬约束,`create_withdraw``IntegrityError` 兜底([`:814-833`](../../../app/repositories/wallet.py))即捕获它。
> 提现单状态机:`reviewing`(待审核,建单即扣现金)→`pending`(审核通过、打款在途)→`success`/`failed`;或 `reviewing``rejected`(已退款)。
### 改动清单(4 处代码 + 1 个迁移)
**① `app/repositories/wallet.py` · `create_withdraw`**
- 删除在途单互斥检查([`:758-765`](../../../app/repositories/wallet.py)):`select WithdrawOrder.id WHERE status IN (_WITHDRAW_ACTIVE_STATUSES)``raise WithdrawTooFrequentError`
- `IntegrityError` 兜底([`:814-833`](../../../app/repositories/wallet.py)):**保留** `out_bill_no` 幂等重试分支([`:816-824`](../../../app/repositories/wallet.py));**删除**其中的在途单二次检查分支([`:825-832`](../../../app/repositories/wallet.py))(索引移除后不会再因在途单触发 `IntegrityError`);末尾其余情况原样 `raise`(仅剩 `out_bill_no` 唯一冲突等,正常不该出现)。
- 删除模块常量 `_WITHDRAW_ACTIVE_STATUSES`([`:38`](../../../app/repositories/wallet.py))与异常类 `WithdrawTooFrequentError`([`:72-73`](../../../app/repositories/wallet.py))。`_NEWBIE_TIER_HELD_STATUSES`(档位资格判定,含 `success`)与本改动无关,**保留**。
**② `app/models/wallet.py` · `WithdrawOrder`**
- 删除 `__table_args__` 中的分区唯一索引 `ux_withdraw_order_user_active`([`:99-107`](../../../app/models/wallet.py))。该表 `__table_args__` 仅此一项,整块移除。
**③ `app/api/v1/wallet.py` · `withdraw` 端点**
- 删除 `except crud_wallet.WithdrawTooFrequentError`([`:225-229`](../../../app/api/v1/wallet.py))这段已失效的 409 处理。
**④ 新增 Alembic 迁移** `alembic/versions/<...>_drop_withdraw_active_unique_index.py`
- `down_revision` = 当前 head(实现时确定)。
- `upgrade`:`op.drop_index("ux_withdraw_order_user_active", table_name="withdraw_order")`
- `downgrade`:`op.create_index("ux_withdraw_order_user_active", "withdraw_order", ["user_id"], unique=True, sqlite_where=text("status IN ('reviewing','pending')"), postgresql_where=text("status IN ('reviewing','pending')"))`,并注释"若已有用户存在多张在途单则重建失败,属预期回滚代价"。
- 索引的 drop/create 为具名操作,SQLite/PG 均无需 `render_as_batch` 重建表。
---
## 5. 天然保持不变(无需改动)的约束
| 约束 | 为何仍成立 |
|---|---|
| **不会超提** | 建单即原子扣现金(`_try_deduct_cash`,余额不足影响 0 行 → `InsufficientCashError`);每笔并行单各需自己的余额 |
| **`coin_cash` 每日档位次数** | `withdraw_tier_states` 的档位闸独立于在途单检查:常规档按**当天发起即计入(任意状态,含被拒/失败,按 `created_at` 北京日)**、每档每日限次(0.5×3 / 10×1 / 20×1)且当天只选一档;新人档(0.1/0.3)按 `reviewing/pending/success` 一次性占用。故即便并行,`coin_cash` 单日在途仍被档位天然封顶(至多 3 笔 0.5) |
| **`out_bill_no` 幂等** | 幂等分支在被删检查之前,同号重试仍原样返回旧单、不重复扣款 |
| **解绑退款** | `refund_reviewing_withdraws_on_unbind``for` 遍历该用户**所有** `reviewing` 单,天然支持多单 |
| **admin 审核 / 打款 / 对账** | `approve_withdraw`/`reject_withdraw`/`refresh_withdraw_status`/对账全按 `out_bill_no` 单号维度,不假设一人一单 |
---
## 6. 客户端/App 团队注意点(跨仓,非本次后端工作)
> - 允许多笔并行后,每次提交都要生成**新的 `out_bill_no`**(复用旧号会命中幂等、返回旧单)。
> - 用户**未开启免确认**时,多笔 `pending` 会各自返回一个微信确认页 `package_info`,App 需能处理/串行多笔待确认。开启免确认后直接到账、无此问题。
---
## 7. 测试计划(`tests/test_withdraw.py`,沿用现有 helper)
- **新增 · 多笔在途放行**:同一用户 seed ≥1 元现金,用**两个不同 `out_bill_no`** 各提 0.5 元(在 0.5 元档 3 次/天限额内)→ 两次均 200 且 `status=reviewing`;`/withdraw-orders` 返回 2 条;余额正确扣两次(1 元 → 0)。
- **新增 · 第二笔余额不足**:seed 0.5 元,连提两笔 0.5 元 → 第一笔 200、第二笔 409(`InsufficientCashError`),验证每笔各需自己的余额。
- **回归 · 幂等**:`test_withdraw_idempotent_same_bill_no`(同 `out_bill_no` → 同一单、只扣一次)仍绿。
- **回归 · 档位限额**:`tests/test_withdraw_tiers.py`(0.5 元日 3 次、第 4 次 409;跨档互斥)不受影响仍绿。
- 沿用 `tests/conftest.py`(临时 SQLite、`RATE_LIMIT_ENABLED=false`);wxpay 网络调用全部 monkeypatch。
---
## 附:涉及文件清单
**改动**
- `app/repositories/wallet.py` — 删在途单检查 + `IntegrityError` 兜底瘦身 + 删 `_WITHDRAW_ACTIVE_STATUSES` / `WithdrawTooFrequentError`
- `app/models/wallet.py` — 删分区唯一索引 `ux_withdraw_order_user_active`
- `app/api/v1/wallet.py` — 删 `WithdrawTooFrequentError` 的 409 处理
- `tests/test_withdraw.py` — 新增多笔在途放行 / 第二笔余额不足用例
**新增**
- `alembic/versions/<...>_drop_withdraw_active_unique_index.py` — drop 分区唯一索引(downgrade 重建)
@@ -0,0 +1,182 @@
# 阿里云短信验证服务 — 设计方案
- 日期:2026-07-25
- 状态:已定稿,待实现
- 范围:新增阿里云 dypns(号码认证服务)短信验证码 provider,与现有极光短信可切换
## 1. 背景与目标
现有短信验证码服务 `app/integrations/sms.py`:本服务**本地生成**验证码、存**进程内存**、极光 REST 仅负责下发;`verify_code()` 比对本地内存(一次性 + 单码失败 `SMS_MAX_VERIFY_ATTEMPTS` 次即作废)。docstring 已标注"内存存码、多 worker 不共享"为技术债。
阿里云文档(`docs/integrations/aliyun/`)为 **号码认证服务 dypns**`SendSmsVerifyCode` + `CheckSmsVerifyCode`:该产品由**阿里云生成并校验**验证码(`{"code":"##code##"}` 模式),核验免费。
目标:接入阿里云该套接口作为一个新的短信 provider,可与极光切换。
## 2. 关键决策(已确认)
1. **验证码模式 = Mode A(阿里云托管码)**:发码用 `SendSmsVerifyCode` + `##code##` 占位符,阿里云生成/存储/下发;校验用 `CheckSmsVerifyCode`,阿里云返回 `PASS/UNKNOWN`。本服务不再本地生成/存储验证码。
2. **可切换 Provider**:新增 `SMS_PROVIDER=jiguang|aliyun` 开关,`send_code/verify_code` 按 provider 分派;保留极光作回退(短信=花钱+登录关键路径,灰度上线/融合认证未开通时可秒切回极光)。
3. **官方 SDK**:调阿里云 dypns 用 `alibabacloud_dypnsapi20170525`,签名/加签由 SDK 处理。
4. **防爆破与极光一致**(排查一致性):阿里云路径**保留**与极光相同的"单码失败 N 次即作废"本地计数,而非改用 API 层频控,避免两 provider 行为不一致导致排查困惑。
## 3. 模块结构(`sms.py` 单文件升级为 provider 包)
```
app/integrations/sms/
__init__.py # 公开 API + 分派器:send_code / verify_code / SmsError
# - SMS_MOCK=true 短路(不碰任何 provider)
# - 按 settings.SMS_PROVIDER 选 jiguang / aliyun
base.py # SmsError(沿用现定义)+ Provider 协议(send_code/verify_code 签名约定)
jiguang.py # 现有自管码逻辑原样迁入(内存存码/冷却/一次性/防爆破 全保留,行为零改动)
aliyun.py # 新增:SendSmsVerifyCode 发码 + CheckSmsVerifyCode 校验 + 本地失败计数
```
- `__init__.py` 继续 re-export `SmsError / send_code / verify_code`,故 `app/api/v1/auth.py:37`
`from app.integrations.sms import SmsError, send_code, verify_code` **导入不变**
- 纯增量重构:极光逻辑整体迁入 `jiguang.py`,对外行为零变化。
## 4. 数据流 — 阿里云 providerMode A
### 4.1 发码 `aliyun.send_code(phone) -> int`
1. 校验 `settings.aliyun_sms_configured`(缺 AK/SignName/TemplateCode → `SmsError(503)`)。
2. 调 `SendSmsVerifyCode`
- `PhoneNumber=phone`
- `SignName=ALIYUN_SMS_SIGN_NAME``TemplateCode=ALIYUN_SMS_TEMPLATE_CODE`
- `TemplateParam = json({"code":"##code##","min": str(ALIYUN_SMS_VALID_TIME_SEC//60)})`
- `CodeLength=ALIYUN_SMS_CODE_LENGTH``ValidTime=ALIYUN_SMS_VALID_TIME_SEC``Interval=ALIYUN_SMS_INTERVAL_SEC`
- `SchemeName=ALIYUN_SMS_SCHEME_NAME`(可空)
3. 成功(`body.Success and body.Code=="OK"`)→ **清本地失败计数**(新码=新预算)→ 返回 `ALIYUN_SMS_INTERVAL_SEC` 作客户端冷却秒数。
4. 失败 → 按 §6 错误码映射抛 `SmsError`
### 4.2 校验 `aliyun.verify_code(phone, code) -> bool`
1. **本地失败计数**`attempts >= SMS_MAX_VERIFY_ATTEMPTS` → 直接 `False`(码已作废,不调阿里云)。
2. 调 `CheckSmsVerifyCode(PhoneNumber, VerifyCode=code, SchemeName)`
3. `body.Model.VerifyResult`
- `"PASS"` → 清计数,返回 `True`(一次性)。
- `"UNKNOWN"``attempts += 1`,返回 `False`(码错/过期)。
4. 网络错误 / 接口非 `OK` → 抛 `SmsError(503)`**不静默返回 False**,区分"阿里云挂了"与"码错了";网络错误不计入 attempts)。
本服务**不存验证码**,仅存一个 per-phone 失败计数(见 §7)。
## 5. 分派器 & mock`__init__.py`
```
send_code(phone):
if settings.SMS_MOCK: # 短路:不碰 provider(测试/开发)
log placeholder code; return cooldown
return _provider().send_code(phone)
verify_code(phone, code):
if settings.SMS_MOCK: # 放行任意 N 位数字(沿用现 mock 语义)
return len(code)==SMS_CODE_LENGTH and code.isdigit()
return _provider().verify_code(phone, code)
_provider(): jiguang if settings.SMS_PROVIDER=="jiguang" else aliyun
```
- mock 语义提到分派层、provider 无关 → 现有 28 个测试文件(conftest 设 `SMS_MOCK=true`)全部零改动通过。
## 6. 错误映射
### 发码(阿里云错误码 → SmsError.status_code
| 阿里云码 | HTTP | 说明 |
|---|---|---|
| `MOBILE_NUMBER_ILLEGAL` | 400 | 手机号格式错误 |
| `BUSINESS_LIMIT_CONTROL` | 429 | 号码天级流控 |
| `FREQUENCY_FAIL` | 429 | 频控(`Interval` 命中) |
| `FUNCTION_NOT_OPENED` | 503 | 融合认证未开通(**critical 日志**,需运维开通) |
| `INVALID_PARAMETERS` | 503 | 参数错误(配置/模板问题,**critical 日志** |
| 其他非 OK / `Success=false` / 网络错误 | 503 | 供应商不可用 |
### 校验
- `PASS` → True`UNKNOWN` → False;接口异常/网络错误 → `SmsError(503)`
## 7. 防爆破 / 频控分工
| 机制 | 极光(Mode B | 阿里云(Mode A |
|---|---|---|
| 验证码存储 | 本地内存 | **阿里云托管**(消除多 worker 存码债) |
| 单号发送冷却 | 本地 `_last_sent` 60s | **交给阿里云 `Interval`**(无本地状态),命中→429 |
| 单设备+IP 频控 | API 层 5/时、20/天 | 同左,**不变** |
| **防爆破(单码失败 N 次即作废)** | 本地 `_CodeRecord.attempts` | **本地 per-phone 计数**,与极光同语义(§4.2 |
- 阿里云路径的**唯一本地状态** = per-phone 失败计数 `dict[phone,int]` + `Lock` + GC(仿极光 `_gc`)。
- 多 worker 降级:失败计数按 worker 各计,effective 上限 = N×workers;与极光现状**同级**(属刻意保留的一致性),且 API 层登录频控(`sms-login-device` 设备+IP 5/时)提供硬兜底。
- 计数复位:`send_code` 成功清计数、`verify` PASS 清计数(新码/验过即新预算)。
- API 层设备频控与测试账号短路(`app/core/test_account.py`**完全不动**。
## 8. 配置项(`app/core/config.py` 新增)
```python
SMS_PROVIDER: str = "jiguang" # jiguang | aliyun;默认极光(保持现状,上线后切 aliyun)
# --- 阿里云 dypns 号码认证 ---
ALIYUN_SMS_ACCESS_KEY_ID: str = ""
ALIYUN_SMS_ACCESS_KEY_SECRET: str = ""
ALIYUN_SMS_SIGN_NAME: str = "" # 系统赠送签名(自定义签名下发易失败)
ALIYUN_SMS_TEMPLATE_CODE: str = "" # 赠送模板 CODE(须与赠送签名搭配)
ALIYUN_SMS_SCHEME_NAME: str = "" # 方案名(可空=默认方案);send/check 必须一致 → 单一来源
ALIYUN_SMS_ENDPOINT: str = "dypnsapi.aliyuncs.com"
ALIYUN_SMS_CODE_LENGTH: int = 6 # CodeLength 4~8
ALIYUN_SMS_VALID_TIME_SEC: int = 300 # ValidTime;短信内 min 文案 = //60
ALIYUN_SMS_INTERVAL_SEC: int = 60 # Interval 单号发送频控
```
- 新增属性 `aliyun_sms_configured`(仿 `mt_cps_configured`):AK_ID/AK_SECRET/SignName/TemplateCode 齐全才为真;`SMS_PROVIDER=aliyun` 但未配 → `send_code``SmsError(503)`
- 复用现有 `SMS_MOCK``SMS_CODE_LENGTH`mock 校验位数)、`SMS_MAX_VERIFY_ATTEMPTS`(防爆破上限,两 provider 共用)。
### 配置敏感点
- `TemplateParam` 变量名(`code`/`min`)须与控制台所选**赠送模板**一致。融合认证验证码模板通常即 `code`+`min`,按此硬编码并加注释;若模板变量名不同,改 `aliyun.py` 该处即可。
- `SchemeName` 在 send 与 check 必须一致,故用**单一** `ALIYUN_SMS_SCHEME_NAME` 供两处,避免不匹配(CheckSmsVerifyCode 文档明确警告)。
## 9. auth.py 改动(最小)
`verify_code` 现在可能抛 `SmsError`(阿里云降级 503)。两处调用点各包一层 `try/except SmsError → HTTPException(e.status_code)`,与 `send_code` 现有写法一致:
- `app/api/v1/auth.py` `sms_login`(约 L185
- `app/api/v1/auth.py` `wechat_bind_phone_sms`(约 L325
`send_code` 调用点已 try/except `SmsError`,无需改。
## 10. 依赖 & SDK
- `pyproject.toml``alibabacloud_dypnsapi20170525`(连带 `alibabacloud-tea-openapi` 等)。
- SDK 同步阻塞调用 → 与现有 sync 端点 + sync httpx 风格一致(FastAPI 跑 threadpool,无碍)。
- `aliyun.py` 内**惰性 import SDK + 惰性建 client**(仿 `wxpay` 惰性加载证书):`SMS_PROVIDER=jiguang` 时不加载 alibabacloud,启动保持精简。
- SDK 调用形态(实现时按实际包名/字段核对):
```python
from alibabacloud_dypnsapi20170525.client import Client
from alibabacloud_dypnsapi20170525 import models as dypns_models
from alibabacloud_tea_openapi import models as open_api_models
cfg = open_api_models.Config(access_key_id=..., access_key_secret=...)
cfg.endpoint = settings.ALIYUN_SMS_ENDPOINT
client = Client(cfg)
resp = client.send_sms_verify_code(dypns_models.SendSmsVerifyCodeRequest(...))
# resp.body.code / resp.body.success / resp.body.model.verify_code
resp = client.check_sms_verify_code(dypns_models.CheckSmsVerifyCodeRequest(...))
# resp.body.model.verify_result == "PASS"
```
## 11. 测试
- 现有测试:`SMS_MOCK=true` → 分派器短路,全绿不变。
- 新增 `tests/test_sms_aliyun.py`monkeypatch SDK client,不发真网络):
1. 发码成功 → 返回 cooldown、清计数。
2. 各错误码 → 对应 `SmsError.status_code`400/429/503)。
3. 校验 `PASS`→True(清计数)/ `UNKNOWN`→False(计数 +1)/ 接口异常→`SmsError(503)`
4. 失败计数达 `SMS_MAX_VERIFY_ATTEMPTS` → 直接 False,不再调阿里云。
5. `send_code` 成功复位计数。
- 新增分派测试:`SMS_PROVIDER` 切换选中正确 provider`SMS_MOCK` 优先于 provider。
## 12. YAGNI(明确不做)
- ❌ 不做 Redis/DB 存码(Mode A 无需;极光路径内存债维持现状,非本次范围)。
- ❌ 不改极光任何行为、不动 API 层频控/测试账号逻辑。
- ❌ 不做多签名/多模板轮换(单签名单模板足够)。
- ❌ 不把失败计数持久化/跨进程(刻意保留与极光同级的本地态)。
## 13. 验收标准
- `SMS_PROVIDER=aliyun` 且配置齐全时:`/sms/send``SendSmsVerifyCode``/sms/login``CheckSmsVerifyCode`,真机可收码并登录。
- `SMS_PROVIDER=jiguang`(默认):行为与当前完全一致。
- `SMS_MOCK=true`:任意 N 位数字通过,不发真短信。
- 阿里云接口异常时:`/sms/login` 返回 503(非 400),日志可区分。
- `ruff check .` 通过;新增/现有 `pytest` 全绿。
@@ -0,0 +1,143 @@
# 创蓝云智(253)短信验证服务 — 设计方案
- 日期:2026-07-26
- 状态:已定稿,待实现
- 范围:新增创蓝云智(253/蓝创云智)模板短信 provider,与现有极光 / 阿里云可切换
## 1. 背景与目标
短信验证码服务已是**可切换 provider** 架构(`app/integrations/sms/``__init__` 分派 + `jiguang` / `aliyun` + `base`)。本次接入第三家 **创蓝云智** 作为新 provider。
创蓝 `tpl/send` v2 接口(调研见 `docs/integrations/chuanglan/tpl-send.md`)是**纯发送网关**:本服务生成验证码、放入 `templateParamJson`,创蓝只负责下发,**无校验接口**。故属 **Mode B(自管码)**,与极光同模式(本地生成/存储/校验),仅"发送调用"不同。
目标:接入创蓝作为可切换 provider;默认仍极光,opt-in 切换,灰度可秒回退。
## 2. 关键决策(已确认)
1. **Mode B 自管码**:本服务 `secrets` 生成 N 位码 → 存进程内存 → 创蓝 REST 只下发;`verify_code` 比对本地存码(一次性 + 失败 `SMS_MAX_VERIFY_ATTEMPTS` 次即作废)。与极光同语义。
2. **代码组织 = 隔离复制(不重构极光)**`chuanglan.py` **自带一份**存码/冷却/校验机器(从 `jiguang.py` 复制适配),**极光文件一行不动**。契合阿里云先例的 provider 隔离哲学,零回归风险于登录关键路径的默认 provider。代价:Mode B 并发逻辑在 jiguang / chuanglan 两处重复,日后改动需同步(YAGNI 权衡,已接受)。
3. **鉴权 = HMAC 签名头**`X-QA-Hmac-Signature`):password 仅用于本地算签、**不上行**。不做明文密码 body 模式。
4. **无新依赖**:创蓝是普通 HTTPS POST,复用现有 `httpx` + 标准库 `hashlib`/`hmac`(对比阿里云需 SDK)。
5. **可切换 + 回退**`SMS_PROVIDER``chuanglan`;默认仍 `jiguang`;误配/未知值一律回退 `jiguang`(保持 `test_unknown_provider_falls_back_to_jiguang` 语义)。
## 3. 模块结构
```
app/integrations/sms/
__init__.py # 分派器: {"aliyun":aliyun,"chuanglan":chuanglan}.get(SMS_PROVIDER, jiguang)
base.py # 不动(SmsError / mock_verify 复用)
jiguang.py # 不动
aliyun.py # 不动
chuanglan.py # 新增(本设计)
```
- `__init__.py` 继续 re-export `SmsError / send_code / verify_code``app/api/v1/auth.py` 导入不变。
- 纯增量:只新增 `chuanglan.py` + 扩分派 dict + 加配置;不改极光/阿里云行为。
## 4. 数据流 — chuanglan providerMode B,复制自 jiguang
### 4.1 发码 `chuanglan.send_code(phone) -> int`
结构与 `jiguang.send_code` 一致:
1. `_lock` 内:`_gc` → 单号冷却检查(`_last_sent``SMS_SEND_INTERVAL_SEC`,命中→`SmsError(429)`)→ `_gen_code()` 生成 N 位 → **预占**(写 `_last_sent` + `_codes[phone]=_CodeRecord(code, expires_at=now+SMS_CODE_TTL_SEC)`)。
2. `_lock` 外:`SMS_MOCK` → 打日志不真发;否则 `_send_via_chuanglan(phone, code)`
3. 失败:**保留冷却**(失败也限速)、`_codes.pop(phone)`(没发出去的码删掉);`SmsError` 原样抛,其他异常 → `SmsError(503)`
4. 返回 `SMS_SEND_INTERVAL_SEC` 作客户端冷却秒数。
### 4.2 校验 `chuanglan.verify_code(phone, code) -> bool`
`jiguang.verify_code` 一致:
- `SMS_MOCK``mock_verify`(放行任意 N 位数字)。
- real`_lock` 内查 `_codes[phone]`;不存在/过期→False(并清);`attempts >= SMS_MAX_VERIFY_ATTEMPTS`→清+False(防爆破);`secrets.compare_digest` 匹配→清+True(一次性);否则 `attempts += 1` 返 False。
### 4.3 发送 `_send_via_chuanglan(phone, code)`(唯一新逻辑)
1. 配置校验 `settings.chuanglan_sms_configured`(缺 account/password/templateId → `SmsError(503)`)。
2. 组装:
- `timestamp = str(int(time.time()))``nonce = secrets.token_hex(16)`32 hex
- `body = {account, timestamp, nonce, phoneNumbers=phone, templateId, templateParamJson=json.dumps([{"param1": code}])}``CHUANGLAN_SMS_SIGNATURE` 非空则加 `signature` 字段。**HMAC 方式 body 不含 password。**
3. 签名 `_sign(password, timestamp, nonce)`
```
md5pwd = md5(password).hexdigest() # 32 位小写 hex
s = "".join(sorted([md5pwd, timestamp, nonce])) # 字典序升序拼接
s = "".join(s.split()) # 去空白(faithful,本例无空白)
sig = hmac_sha256(key=md5pwd.encode(), msg=s.encode()).hexdigest() # 小写 hex
```
置请求头 `X-QA-Hmac-Signature: sig``Content-Type: application/json`
4. `httpx.post(CHUANGLAN_SMS_ENDPOINT, json=body, headers=..., timeout=CHUANGLAN_SMS_TIMEOUT_SEC)`;网络异常 → `SmsError(503)`
5. 解析 `resp.json()``code == "000000"` → 成功返回;否则按 §5 映射抛 `SmsError`。HTTP≠200 或 JSON 解析失败 → `SmsError(503)`
## 5. 错误码映射(创蓝 `code` → SmsError.status_code
| 创蓝 code | HTTP | 处理 |
|---|---|---|
| `000000` | — | 成功 return |
| `103` | 429 | 超频,"发送过于频繁,请稍后再试" |
| `107` | 400 | 手机号错误,"手机号无效" |
| `109` | 503 | 无发送量/余额 → **critical 日志**(需充值) |
| `117` | 503 | IP 未白名单 → **critical 日志**(需运维加白) |
| `102` / `116` / `124` / `152` / `101` / `118` | 503 | 密码/签名/模板/账号/权限配置错 → **critical 日志** |
| 其他 / `Success` 非 000000 / HTTP≠200 / 网络错 | 503 | "短信服务暂不可用,请稍后重试" |
- 映射用 `_SEND_ERRORS: dict[str,(int,str)]` + `_SEND_CRITICAL_CODES: frozenset`(仿 aliyun 写法)。
## 6. 防爆破 / 频控分工(与极光同级)
| 机制 | 实现 |
|---|---|
| 验证码存储 | 本地进程内存 `_codes`(与极光同,多 worker 不共享的技术债同级) |
| 单号发送冷却 | 本地 `_last_sent``SMS_SEND_INTERVAL_SEC` |
| 单设备+IP 频控 | API 层(`app/api/v1/auth.py`),**不变** |
| 防爆破(单码失败 N 次作废)| 本地 `_CodeRecord.attempts``SMS_MAX_VERIFY_ATTEMPTS` |
- 创蓝控制台侧另建议叠加:**IP 白名单**(否则 117)+ 发送频控。
## 7. 配置项(`app/core/config.py` 新增)
```python
SMS_PROVIDER: Literal["jiguang", "aliyun", "chuanglan"] = "jiguang"
# --- 创蓝云智(253)模板短信,Mode B 自管码,httpx 直连 + HMAC 签名 ---
CHUANGLAN_SMS_ACCOUNT: str = "" # YZM 前缀验证码账号
CHUANGLAN_SMS_PASSWORD: str = "" # API 密码(仅本地算签,不上行)
CHUANGLAN_SMS_TEMPLATE_ID: str = "" # 模板 ID
CHUANGLAN_SMS_SIGNATURE: str = "" # 短信签名文案【品牌】;模板已带签名则留空
CHUANGLAN_SMS_ENDPOINT: str = "https://smssh.253.com/msg/sms/v2/tpl/send"
CHUANGLAN_SMS_TIMEOUT_SEC: int = 10 # httpx 读/连超时
```
- 新增属性 `chuanglan_sms_configured`(仿 `aliyun_sms_configured`):account/password/templateId 齐全才为真。
- 复用 `SMS_MOCK` / `SMS_CODE_LENGTH` / `SMS_CODE_TTL_SEC` / `SMS_SEND_INTERVAL_SEC` / `SMS_MAX_VERIFY_ATTEMPTS`provider 无关的 Mode B 旋钮)。
- `.env.example``CHUANGLAN_SMS_*` 块 + 注释。
### 模板变量敏感点
- 默认按**单占位** `templateParamJson=[{"param1": code}]`(模板形如「您的验证码 {s},5分钟内有效」)。
- 若控制台模板把「有效分钟」也做成第二个 `{s}`,实现时在此加 `param2`(改 `chuanglan.py` 一处)。
## 8. auth.py 改动
无。`send_code` / `verify_code` 签名与返回不变,分派层内部路由;两调用点现有 `try/except SmsError` 已覆盖 chuanglan 的 429/400/503。
## 9. 测试
- 现有测试:`SMS_MOCK=true` → 分派器短路,全绿不变。
- 新增 `tests/test_sms_chuanglan.py`monkeypatch `_send_via_chuanglan` 内 httpx 接缝,不发真网络):
1. 发码成功(`code=000000`)→ 返回 cooldown、码入内存。
2. 各错误码 → 对应 `SmsError.status_code`103→429 / 107→400 / 109/117/其他→503)。
3. HTTP≠200 / 网络异常 → `SmsError(503)`
4. 校验:匹配→True 且作废(一次性);过期→False;失败累计达上限→作废 False;不匹配→attempts+1 False。
5. 冷却:`SMS_SEND_INTERVAL_SEC` 内二次发 → `SmsError(429)`
6. `_sign` 签名算法:对固定 (password, ts, nonce) 断言 HMAC 输出(独立复算比对)。
- 扩 `tests/test_sms_dispatch.py``SMS_PROVIDER=chuanglan` 路由命中 chuanglan;未知值回退 jiguang。
## 10. YAGNI(明确不做)
- ❌ 不重构极光 / 不动阿里云。
- ❌ 不做明文密码 body 模式(只 HMAC)。
- ❌ 不做状态回执 `report` / `callbackUrl`
- ❌ 不做批量发送(验证码单号)。
- ❌ 不做 DB/Redis 存码(与极光同级内存态,多 worker 债维持现状)。
## 11. 验收标准
- `SMS_PROVIDER=chuanglan` 且配置齐全时:`/sms/send` 走创蓝 `tpl/send``/sms/login` 本地校验,真机可收码并登录。
- `SMS_PROVIDER=jiguang`(默认)/ `aliyun`:行为与当前完全一致。
- `SMS_MOCK=true`:任意 N 位数字通过,不真发。
- 创蓝接口异常时:`/sms/send` 返回对应码(429/400/503),日志可区分(余额/白名单打 critical)。
- `ruff check .` 通过;新增/现有 `pytest` 全绿。

Some files were not shown because too many files have changed in this diff Show More