Compare commits

..

25 Commits

Author SHA1 Message Date
guke 216441a401 feat(observe): 生产部署硬化 + nginx 反代(Certbot+BasicAuth) + 恢复采集中间件
- app/main.py: 恢复 merge 丢失的 RequestMetricsMiddleware 注册,接口埋点重新生效
- docker-compose.prod.yml: 镜像 pin v0.91.2、数据保留 30 天、关匿名遥测(ZO_TELEMETRY)、容器日志上限、资源硬上限(2C/3G)、数据改 bind-mount 到宿主 /data
- observe.shaguabijia.com.conf: 改用 Certbot 证书(与 admin-web 一致)+ Basic Auth 兜底(办公网无固定 IP)+ 去掉与 admin-web 冲突的 ipv6only=on
- dashboard-api-metrics.json: 补 dashboardId 修复新版 OpenObserve 导入校验
- app-api.conf: client_max_body_size 4m->32m,修带截图反馈被 nginx 413(顺带)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-20 18:47:43 +08:00
guke 7ed942cb8c Merge branch 'main' of https://gitea.shaguabijia.com/WonderableAI/shaguabijia-app-server into feat/openobserve-api-metrics
# Conflicts:
#	app/main.py
2026-07-20 10:16:54 +08:00
guke 130a7dff29 docs(welfare): 15天不活跃清零金币/现金 设计文档(spec) (#144)
修改INACTIVITY_RESET_ENABLED语义,为false时清空金币操作只记录审计日志,不执行操作。

---------

Co-authored-by: guke <guke@autohome.com.cn>
Reviewed-on: #144
2026-07-19 00:14:03 +08:00
guke a86688ccfb feat(welfare): 15 天不活跃清零(金币+折算现金,邀请金不清)+ 预警 + admin 活跃口径统一 (#143)
背景 / 目标
连续 15 天(北京自然日)没有「首页可见 / 发起比价 / 发起领券」行为的用户判定为流失,每日自动清零其金币 + 折算现金;清零前按可配置节奏预警;全程留审计。纯登录不算活跃;邀请奖励金物理隔离、不清(产品红线)。

改了什么
活跃口径共享模块 app/repositories/activity.py:worker(清零/预警)与 admin(最近活跃)单一真源,防漂移。
清零业务逻辑 inactivity.py:选取 / 逐用户清零(行锁 + 幂等)/ 预警分档 + streak 去重 / run_once 组合,预警故障逐用户隔离、绝不阻塞清零。
每日 worker inactivity_reset_worker.py(仿 daily_exchange:文件锁 + 北京日守卫 + RUN_HOUR 门槛 + 总闸)+ main.py lifespan 接线。
可插拔通知器 notifier.py(v1 LogNotifier 日志占位,预留 JPush/短信)。
配置 INACTIVITY_*(阈值 / 预警档 / 执行点 / 通道 / 开关)。
admin 最近活跃口径改用共享模块(移除 last_login_at、纳入 show/home、以 created_at 为基线)。
审计:inactivity_reset_log + inactivity_notification_log 两表 + 钱包流水双写(biz_type=inactivity_reset,ref_id 交叉)。
文档:设计 spec / 实现 plan / docs/database/ 两表字典。
关键产品决策
邀请金不清:只清金币 + 折算现金,invite_cash_balance_cents 原封(仅快照入审计)。
活跃口径 = max(created_at, 首页可见, 比价, 领券),不含 last_login_at;首页可见 = event=show + page=home。
时间边界:北京自然日 0 点对齐(见 activity.reset_cutoff)。
总闸默认关,灰度验证后再开。
数据库变更
新表:inactivity_reset_log、inactivity_notification_log。
analytics_event 新增复合覆盖索引 ix_analytics_event_active (event, page, user_id, created_at)(活跃口径聚合热点)。
修复了 base 上的迁移多头(135e79414fd0 与 phone_rebind_log 同从 comparison_llm_cost 分叉)→ 加空 merge 修订,alembic upgrade head 恢复单头正常。
⚠️ 上线注意(合并后 / 开总闸前)
INACTIVITY_RESET_ENABLED 默认 false;开启前提 = show/home 埋点全量铺满——否则"只登录不操作"且注册满 15 天的老用户会落到 created_at 基线被误清。
前端依赖:Android 端需在首页可见上报 event=show + page=home(携带登录后的 user_id)。
admin「最近活跃」口径变化(去登录、纳入 home_view、created_at 基线):属预期变化,需产品/运营知会;与 DAU(_period_active_user_ids,仍含登录)是两套指标。
清零对 C 端「金币流水」可见(biz_type=inactivity_reset,备注「15天不活跃清零」)。
灰度:先只看预警/清零名单对不对,再开总闸。

---------

Co-authored-by: guke <guke@autohome.com.cn>
Reviewed-on: #143
2026-07-18 19:11:45 +08:00
guke 061f6baaf1 feat(auth): 微信登录与手机号绑定 / 占用冲突处理(M2 + M3 §10) (#139)
## 背景 / 需求

新增「微信登录」链路:用户用微信授权登录 App。openid 已绑账号即直接登入;
未绑则走「绑手机号」建号,并处理「手机号已被别的账号占用」的冲突(M2),
以及绑微信后展示昵称/头像的替换策略(M3 §10)。

关键取舍:微信登录**不套用**钱包提现里 bind-wechat 的「撞号即 409」逻辑
——登录场景 openid 命中就应登入,不能因撞号把用户挡在门外。

## 改动概览:新增 5 个端点(前缀 `/api/v1/auth`)

| 方法 | 路径 | 作用 |
|---|---|---|
| POST | `/wechat-login` | code→openid。命中已绑用户→直接登入;未命中→签发 `bind_ticket`,返回 `need_bind_phone`(**此刻不建号**);未配 APP_ID/SECRET→503 |
| POST | `/wechat/bind-phone/sms` | 持 `bind_ticket` + 手机号 + 短信码绑号 |
| POST | `/wechat/bind-phone/jverify` | 持 `bind_ticket` + 极光本机号一键取号绑号 |
| POST | `/wechat/conflict/continue` | 占用冲突·继续绑定=**登录老账号**(老号没绑微信则把 openid 绑上;已绑别的微信则只登入、丢弃本次 openid) |
| POST | `/wechat/conflict/rebind` | 占用冲突·换绑=**单事务**注销老账号 X(腾号)+ 用该号重建新账号 Y + 写换绑台账;受 30 天限制 |

绑号两条取号路径(sms / jverify)尾段共用 `_finish_wechat_bind`:
手机号未注册→建微信账号(channel=wechat,昵称头像取微信)登入;
已被占用→返回 `phone_occupied`(带占用账号昵称/头像/注册时间/`has_wechat`、
`conflict_ticket`、`rebind_available`、`rebind_blocked_days`),交前端冲突页。
冲突页三选一,后端提供 continue / rebind 两个动作(「取消」为前端本地行为)。

## 关键设计

**两种短时令牌(`core/security.py`,复用 `JWT_SECRET_KEY`,靠 `typ` 区分):**
- `bind_ticket`(typ=wechat_bind,sub=openid,附微信昵称/头像,TTL 10min):
  openid 未命中时下发,覆盖「授权→输手机号→收码→验码」整个绑定流程。
- `conflict_ticket`(typ=wechat_conflict):比 bind_ticket **多编码已验证的手机号**;
  换绑 / 继续绑定只认票里的 phone,堵住「拿别人手机号去夺号」的接管漏洞。

**30 天换绑限制:** 新增台账表 `phone_rebind_log`(手机号级、渠道无关),
记录「腾号重建」这一破坏性事件,靠 `rebound_at` 算窗口;
阈值走配置 `PHONE_REBIND_LIMIT_DAYS`。命中限制的换绑请求返回 409。

**M3 §10 昵称/头像替换:** 绑微信默认用微信昵称/头像替换展示,
抽成共享 helper,登录绑号路径与钱包绑微信路径两处共用
(故 `wallet.py`、`tests/test_withdraw.py` 一并有改动)。

---------

Co-authored-by: guke <guke@autohome.com.cn>
Reviewed-on: #139
2026-07-17 21:49:10 +08:00
liujiahui b395648b7c fix(withdraw): 新人档提现没成功也被判定已用(解绑微信退回后 0.1 永久锁死) (#142)
## 现象

新人提现 0.1 元,提现审核中时解绑微信 —— 现金正常退回,但「新人领取的 0.1 元」被判定为已使用:该档位从提现页永久消失,重提报 400。

## 根因

新人档(0.1/0.3)「一次性」的判定口径只看**有没有发起过**,完全不看提现单的最终状态。`withdraw_tier_states()` 里的 `used_newbie` 查询没有任何 `status` 过滤:

```python
select(WithdrawOrder.amount_cents).distinct().where(
    WithdrawOrder.user_id == user_id,
    WithdrawOrder.source == "coin_cash",
    WithdrawOrder.amount_cents.in_(newbie_amounts),
    # ← 没有 status 条件:只要「存在这条单」就算用过
)
```

而解绑微信时,`refund_reviewing_withdraws_on_unbind` → `_refund_withdraw` 把待审核单**退回现金 + 置 `failed`**,但记录仍在库里。两者相撞:那条 `failed` 的单照样落进 `used_newbie` → **钱退了,资格没退**。

## 改法

判定口径从「发起过没有」改为「钱是不是真到手 / 正在路上」——只认 `reviewing`/`pending`/`success`:

| status | 含义 | 钱在哪 | 新人档 |
|---|---|---|---|
| `reviewing` / `pending` | 待审核 / 打款在途 | 已扣,挂在单上 | 占用(防重复发起) |
| `success` | 打款成功 | **进了用户微信** | 用掉,永久锁死 |
| `failed` | 转账失败 / **解绑退回** | 已退回余额 | **恢复可提** |
| `rejected` | 审核拒绝 | 已退回余额 | **恢复可提** |

核心即查询加一行 `WithdrawOrder.status.in_(_NEWBIE_TIER_HELD_STATUSES)`。

- 下发(`withdraw-info`)与下单校验(`create_withdraw`)共用 `withdraw_tier_states`,**改一处两处同时生效**,不会出现「显示能提、点了 400」。
- 复用现有 `status` 语义,**无新字段、无 alembic 迁移**。
- **存量受害用户上线后自动恢复**(他们那条单是 `failed`,新查询天然排除它),无需数据修复脚本;已成功提现过的(`success`)仍正确锁死,不会误恢复。

## ⚠️ 行为变更

7-9 定的「新人档发起就算(含被拒/失败)」按本次拍板**放宽为「没成功打款就恢复」**。

常规档(0.5 / 10 / 20)的「当天名额发起就算、被拒不退」规则**一行未动**。

## 🔴 测试按要求不进本 PR —— 但既有测试会失败,合入前需同步处理

本 PR 只含源码改动。**既有测试 `tests/test_withdraw_tiers.py::test_newbie_tiers_independent_and_once_forever` 锁死的正是本次推翻的旧口径**(它断言「被拒后 0.1 消失、重提 400」)。

已实测(main 上的原版测试 + 本 PR 的新代码):

```
FAILED tests/test_withdraw_tiers.py::test_newbie_tiers_independent_and_once_forever
1 failed, 5 passed
```

**本 PR 合入前需同步调整该测试,否则 main 会红。** 本地已备好可用的测试改动(反转该断言,另加「解绑退回后恢复」回归 + 「成功打款后永久锁死」),需要即可单独提供。

## 待办

- 上述既有测试需同步调整(否则合入后 main 红)
- 纯后端逻辑改动,**未真机验证**

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: no_gen_mu <liujianhishen@gmail.com>
Reviewed-on: #142
Co-authored-by: liujiahui <liujiahui@wonderable.ai>
Co-committed-by: liujiahui <liujiahui@wonderable.ai>
2026-07-17 16:34:55 +08:00
zhuzihao e9fd51d119 fix(auth): 发码防刷改为按成功计数 + 新增每设备每日发码上限 (#136)
- 修复:发码限流原为原子「判+记」,被单号 60s 冷却挡下的重发也占设备额度
  → 正常用户连点重发可能被误锁 1 小时。改为「先判后记、只对成功发码计数」:
  check 判在真发之前(超限直接 429、不真发),record 只在 send_code 成功后调;
  被单号冷却 / 供应商失败抛 429 时直接返回、不计数。
- 新增:同一设备(device_id)+ IP 每天最多 20 次发码上限,与原每小时 5 次两道闸并存,
  均按成功计数,叠一层日封顶挡低频长时间轰炸。
- 基建:ratelimit.py 新增 RateLimitRule + check_rate_limits / record_rate_limits
  (peek/commit 拆分);原子的 enforce_rate_limit 仍保留给登录爆破(失败也计)不变。
- 测试:补 2 个用例(冷却挡下不占额度 / 每日上限)。

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: zzhyyyyy <2685922758@qq.com>
Reviewed-on: #136
Co-authored-by: zhuzihao <zhuzihao@wonderable.ai>
Co-committed-by: zhuzihao <zhuzihao@wonderable.ai>
2026-07-16 09:40:56 +08:00
guke e0874112a9 Merge branch 'main' of https://gitea.shaguabijia.com/WonderableAI/shaguabijia-app-server into feat/openobserve-api-metrics 2026-07-06 15:15:45 +08:00
guke 00404c8b45 feat(observe): 生产部署(单机)—— 硬化 compose + nginx 反代 + README(SSH隧道/保留期/专用账号)
- docker-compose.prod.yml: 只绑 127.0.0.1、命名卷、mem 1g、密码走 .env(:? 守卫)
- nginx/observe.shaguabijia.com.conf: 子域名反代 + IP白名单 + TLS + WS 透传
- .gitignore: 忽略 prod 密码 .env
- README: 生产部署步骤 + UI 访问两方案(SSH隧道推荐)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 14:38:26 +08:00
guke 964032d16f feat(observe): 现成 OpenObserve 仪表盘(QPS/P95/分位/错误率) + 导入说明
dashboard-api-metrics.json: v8 schema 4 面板,已在 v0.91.1 实例 test-import 验证可用。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 13:39:39 +08:00
guke 1f97b19d21 observe: 关停时补记残留丢弃计数(终审 minor:账不丢在关停期)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 12:53:34 +08:00
guke 8938b951fb feat(observe): OpenObserve 本地 compose + README(查询/仪表盘) + .env.example
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 12:17:28 +08:00
guke 0cf4cc706b feat(observe): main.py 挂中间件 + lifespan 启停上报 worker
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 12:14:14 +08:00
guke a5f63cb53c harden(observe): 防重复 start 泄漏 client + 关停丢余量注释 + 补 stop/失败存活测试
评审建议(Task4 code-quality):
- start 已启动则不重复建 client(避免泄漏旧连接池)
- 关停只发一批、超出丢弃,补注释说明 best-effort
- 补测: _run_loop 失败后存活 / stop drain+发最后一批+关 client

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 12:11:16 +08:00
guke 65e9b422fe fix(observe): 保留 asyncio.TimeoutError 捕获(3.10 兼容,撤销 UP041 自动改写)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 12:04:25 +08:00
guke 0350a140fe feat(observe): 加后台批量上报 worker(失败丢批不重试)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 12:03:06 +08:00
guke 95a4c5c3cc feat(observe): 加 RequestMetricsMiddleware(路由模板+状态码+耗时) 2026-07-06 11:33:13 +08:00
guke 822c2ca2a6 docs(plan): 去注解引号 + Task7 lint 只查改动文件(基线558既有ruff错)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 11:16:00 +08:00
guke 9354588ba2 style(observe): 去冗余注解引号 + 整理 import(ruff UP037/I001)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 11:14:52 +08:00
guke 6c624c3cea feat(observe): 加有界事件队列与 record_event(满则丢) 2026-07-06 11:04:00 +08:00
guke 70c5c4cf08 docs(plan): 拆分 observe.py 导入,避免 Task 2 提交时 ruff F401
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 10:58:28 +08:00
guke 804de02188 test(observe): 补 observe_configured 缺用户名分支断言
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 10:56:38 +08:00
guke 76cb981d84 feat(observe): 加 OBSERVE_* 配置与 observe_configured 门槛
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-06 10:48:27 +08:00
guke 20fa32e884 docs: OpenObserve 接口指标实现计划(分步 TDD)
7 个任务:配置门槛 → 事件队列/record_event → 中间件 → 上报 worker →
main 接线 → OpenObserve compose/README/.env → 端到端验证。每任务 TDD + 提交。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 10:40:24 +08:00
guke 29aa6fbff8 docs: OpenObserve 接口 QPS/耗时可观测设计 spec
方案 A:轻量自研 ASGI 中间件 + 后台 worker 批量直采到本地 Docker OpenObserve。
仅 app-server(8770);默认关、opt-in;队列满丢弃、上报失败不重试、跳过 /health。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 09:50:18 +08:00
61 changed files with 6740 additions and 1079 deletions
+16
View File
@@ -137,3 +137,19 @@ PANGLE_REPORT_SECURITY_KEY=
# GroMore AppId(报表 site_id 维度)→ 应用环境;默认取现网两个应用,按需覆盖。
PANGLE_REPORT_SITE_ID_PROD=5830519
PANGLE_REPORT_SITE_ID_TEST=5832303
# ===== 可观测(OpenObserve 接口指标)=====
# 采集每个接口 QPS + 耗时 + 错误率,批量直采到 OpenObserve(本地 Docker,见 deploy/openobserve/)。
# 默认关;开启需 ENABLED=true 且填 USER/PASSWORD(与 docker-compose 里 root 账号一致)。
# 未开/缺凭证 → 中间件透传、worker 不启动,整套 no-op,不影响业务。
OBSERVE_ENABLED=false
OBSERVE_ENDPOINT=http://localhost:5080
OBSERVE_ORG=default
OBSERVE_STREAM=app_requests
OBSERVE_USER=admin@shaguabijia.local
OBSERVE_PASSWORD=Complexpass#123
# 进阶(一般不用改):攒批间隔秒 / 单批最大条数 / 有界队列上限(满则丢) / 上报超时秒
OBSERVE_FLUSH_INTERVAL_SEC=5
OBSERVE_BATCH_MAX=200
OBSERVE_QUEUE_MAX=10000
OBSERVE_TIMEOUT_SEC=5
@@ -0,0 +1,68 @@
"""add inactivity tables
Revision ID: 135e79414fd0
Revises: comparison_llm_cost
Create Date: 2026-07-16 18:31:02.105929
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = '135e79414fd0'
down_revision: Union[str, Sequence[str], None] = 'comparison_llm_cost'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.create_table(
"inactivity_reset_log",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("user_id", sa.Integer(), nullable=False),
sa.Column("coin_balance_before", sa.Integer(), nullable=False),
sa.Column("cash_balance_cents_before", sa.Integer(), nullable=False),
sa.Column("invite_cash_balance_cents_before", sa.Integer(), nullable=False),
sa.Column("last_active_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("inactive_days", sa.Integer(), nullable=False),
sa.Column("reason", sa.String(length=32), nullable=False),
sa.Column("reset_at", sa.DateTime(timezone=True),
server_default=sa.text("(CURRENT_TIMESTAMP)"), nullable=False),
sa.PrimaryKeyConstraint("id"),
)
with op.batch_alter_table("inactivity_reset_log", schema=None) as batch_op:
batch_op.create_index(batch_op.f("ix_inactivity_reset_log_user_id"), ["user_id"], unique=False)
batch_op.create_index(batch_op.f("ix_inactivity_reset_log_reset_at"), ["reset_at"], unique=False)
op.create_table(
"inactivity_notification_log",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("user_id", sa.Integer(), nullable=False),
sa.Column("stage", sa.Integer(), nullable=False),
sa.Column("inactive_days", sa.Integer(), nullable=False),
sa.Column("coin_balance", sa.Integer(), nullable=False),
sa.Column("cash_balance_cents", sa.Integer(), nullable=False),
sa.Column("invite_cash_balance_cents", sa.Integer(), nullable=False),
sa.Column("channel", sa.String(length=16), nullable=False),
sa.Column("status", sa.String(length=16), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True),
server_default=sa.text("(CURRENT_TIMESTAMP)"), nullable=False),
sa.PrimaryKeyConstraint("id"),
)
with op.batch_alter_table("inactivity_notification_log", schema=None) as batch_op:
batch_op.create_index(batch_op.f("ix_inactivity_notification_log_user_id"), ["user_id"], unique=False)
batch_op.create_index(batch_op.f("ix_inactivity_notification_log_created_at"), ["created_at"], unique=False)
def downgrade() -> None:
with op.batch_alter_table("inactivity_notification_log", schema=None) as batch_op:
batch_op.drop_index(batch_op.f("ix_inactivity_notification_log_created_at"))
batch_op.drop_index(batch_op.f("ix_inactivity_notification_log_user_id"))
op.drop_table("inactivity_notification_log")
with op.batch_alter_table("inactivity_reset_log", schema=None) as batch_op:
batch_op.drop_index(batch_op.f("ix_inactivity_reset_log_reset_at"))
batch_op.drop_index(batch_op.f("ix_inactivity_reset_log_user_id"))
op.drop_table("inactivity_reset_log")
@@ -0,0 +1,36 @@
"""analytics_event 活跃口径复合索引
Revision ID: analytics_active_idx
Revises: 135e79414fd0
Create Date: 2026-07-18 17:35:00.000000
给 analytics_event 加活跃口径热点复合索引 (event, page, user_id, created_at):
activity.active_event_condition 按 (event=show & page=home) 比价 领券 过滤后
group by user_id、max(created_at)。覆盖索引让该聚合走 index-only,避免高频 show 事件全表扫。
⚠️ 本分支迁移树有**既有多头**:135e79414fd0(不活跃两表)与 phone_rebind_log 同从
comparison_llm_cost 分叉,`alembic upgrade head` 会多头报错。本迁移挂在 135e79414fd0
一侧;集成到 main 时需 `alembic merge` 合并 phone_rebind_log 那个头(与本迁移无关的既有问题)。
"""
from typing import Sequence, Union
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "analytics_active_idx"
down_revision: Union[str, Sequence[str], None] = "135e79414fd0"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.create_index(
"ix_analytics_event_active",
"analytics_event",
["event", "page", "user_id", "created_at"],
unique=False,
)
def downgrade() -> None:
op.drop_index("ix_analytics_event_active", table_name="analytics_event")
@@ -0,0 +1,25 @@
"""merge inactivity(analytics_active_idx) + phone_rebind_log heads
Revision ID: merge_active_phone
Revises: analytics_active_idx, phone_rebind_log
Create Date: 2026-07-18 18:52:34.001148
"""
from typing import Sequence, Union
from alembic import op
# revision identifiers, used by Alembic.
revision: str = 'merge_active_phone'
down_revision: Union[str, Sequence[str], None] = ('analytics_active_idx', 'phone_rebind_log')
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
pass
def downgrade() -> None:
pass
+32
View File
@@ -0,0 +1,32 @@
"""phone_rebind_log 表(M2 换绑 30 天限制台账)
Revision ID: phone_rebind_log
Revises: comparison_llm_cost
"""
from alembic import op
import sqlalchemy as sa
revision = "phone_rebind_log"
down_revision = "comparison_llm_cost"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.create_table(
"phone_rebind_log",
sa.Column("id", sa.Integer(), primary_key=True, autoincrement=True),
sa.Column("phone", sa.String(length=20), nullable=False),
sa.Column("old_user_id", sa.Integer(), nullable=True),
sa.Column("new_user_id", sa.Integer(), nullable=False),
sa.Column("source", sa.String(length=32), nullable=False, server_default="wechat_conflict"),
sa.Column("rebound_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
)
op.create_index("ix_phone_rebind_log_phone", "phone_rebind_log", ["phone"])
op.create_index("ix_phone_rebind_log_rebound_at", "phone_rebind_log", ["rebound_at"])
def downgrade() -> None:
op.drop_index("ix_phone_rebind_log_rebound_at", table_name="phone_rebind_log")
op.drop_index("ix_phone_rebind_log_phone", table_name="phone_rebind_log")
op.drop_table("phone_rebind_log")
+11 -62
View File
@@ -11,7 +11,6 @@ from zoneinfo import ZoneInfo
from sqlalchemy import Select, asc, case, desc, func, or_, select
from sqlalchemy.orm import Session
from app.admin.repositories.stats import COMPARE_START_EVENT, COUPON_START_EVENT
from app.core import rewards
from app.core.config import settings
from app.models.ad_feed_reward import AdFeedRewardRecord
@@ -32,10 +31,7 @@ from app.models.wallet import (
InviteCashTransaction,
WithdrawOrder,
)
from app.repositories import ad_ecpm
# 「最近活跃」计入的行为事件(与大盘 DAU/留存活跃口径一致:开始比价 + 开始领券)
_ACTIVE_EVENTS = (COMPARE_START_EVENT, COUPON_START_EVENT)
from app.repositories import activity, ad_ecpm
# 折算成可提现现金时,非广告金币来源的排除集(广告单独统计、人工调整不算"赚取")
_NON_TASK_BIZ_TYPES = ("reward_video", "feed_ad_reward", "admin_grant", "admin_deduct")
@@ -88,49 +84,6 @@ def offset_paginate(
return items, next_cursor, total
def _last_active_parts():
"""「最近活跃」的两个按 user_id 预聚合派生表(最近开始比价/领券事件、最近领券发起)。
活跃口径与大盘 DAU/留存一致(2026-07-05 产品定:进入 App≈登录 last_login_at +
发起比价 real_compare_start + 发起领券 real_coupon_start/claim_started)。
用 LEFT JOIN 预聚合而非相关标量子查询:后者在 PG 上对 users 每行各跑一个 SubPlan
(排序键、range 筛选、offset_paginate 的 count 三处叠加),埋点表大了会拖垮列表接口;
预聚合借 analytics_event.event 索引只扫两类 start 事件,每次查询聚合一次。
"""
ev_agg = (
select(
AnalyticsEvent.user_id.label("user_id"),
func.max(AnalyticsEvent.created_at).label("last_at"),
)
.where(
AnalyticsEvent.user_id.is_not(None),
AnalyticsEvent.event.in_(_ACTIVE_EVENTS),
)
.group_by(AnalyticsEvent.user_id)
.subquery()
)
eng_agg = (
select(
CouponPromptEngagement.user_id.label("user_id"),
func.max(CouponPromptEngagement.created_at).label("last_at"),
)
.where(
CouponPromptEngagement.user_id.is_not(None),
CouponPromptEngagement.engage_type == "claim_started",
)
.group_by(CouponPromptEngagement.user_id)
.subquery()
)
return ev_agg, eng_agg
def _norm_utc(dt: datetime | None) -> datetime | None:
"""naive 视为 UTC 补 tzinfo(SQLite 读回 naive、PG 读回 aware,混着 max() 会 TypeError)。"""
if dt is None:
return None
return dt if dt.tzinfo is not None else dt.replace(tzinfo=timezone.utc)
def _attach_last_active(db: Session, users: list[User]) -> None:
"""给本页用户瞬态挂 last_active_at(非 DB 列,供 AdminUserListItem from_attributes 读)。
@@ -144,7 +97,7 @@ def _attach_last_active(db: Session, users: list[User]) -> None:
select(AnalyticsEvent.user_id, func.max(AnalyticsEvent.created_at))
.where(
AnalyticsEvent.user_id.in_(uids),
AnalyticsEvent.event.in_(_ACTIVE_EVENTS),
activity.active_event_condition(),
)
.group_by(AnalyticsEvent.user_id)
).all()
@@ -161,9 +114,9 @@ def _attach_last_active(db: Session, users: list[User]) -> None:
)
for u in users:
candidates = [
_norm_utc(u.last_login_at),
_norm_utc(ev_map.get(u.id)),
_norm_utc(eng_map.get(u.id)),
activity.norm_utc(u.created_at), # baseline 由 last_login_at 改为 created_at(登录不算活跃)
activity.norm_utc(ev_map.get(u.id)),
activity.norm_utc(eng_map.get(u.id)),
]
u.last_active_at = max((c for c in candidates if c is not None), default=None)
@@ -191,16 +144,12 @@ def list_users(
(口径见 [_last_active_expr])。**offset 分页**(cursor=offset):任意列排序下游标语义统一,
代价是翻页期间数据变动可能错位一条——admin 低频场景可接受(同 [list_all_withdraw_orders])。
日期入参统一转 tz-aware UTC 比较(列为 timestamptz,见 _as_utc)。"""
# 最近活跃 = max(最近登录, 最近行为事件, 最近领券发起)。PG 用 GREATEST;SQLite 标量 max()
# 任一参数 NULL 即返回 NULL,故 LEFT JOIN 未命中侧 coalesce 到 last_login_at 兜底
# (注册即登录,该列恒非空)。派生表 1:1(按 user_id 聚合),outerjoin 不放大行数,
# offset_paginate 的 count 不受影响。
ev_agg, eng_agg = _last_active_parts()
greatest = func.greatest if db.get_bind().dialect.name == "postgresql" else func.max
last_active = greatest(
User.last_login_at,
func.coalesce(ev_agg.c.last_at, User.last_login_at),
func.coalesce(eng_agg.c.last_at, User.last_login_at),
# 最近活跃 = max(注册时间, 最近行为事件, 最近领券发起)。baseline 由 last_login_at 改为 created_at
#(登录不代表在用 App;口径统一到 activity.py,含 home_view + 比价 + 领券,见 activity.ACTIVE_EVENTS)。
# 未命中侧 coalesce 到 created_at(恒非空基线)。派生表 1:1,outerjoin 不放大行数
ev_agg, eng_agg = activity.last_active_subqueries(db)
last_active = activity.last_active_expr(
User.created_at, ev_agg, eng_agg, db.get_bind().dialect.name
)
stmt = (
select(User)
-58
View File
@@ -1,58 +0,0 @@
"""会话存档独立 poller → app-server 的内部信号端点(server→server, 非客户端接口)。
会话存档轮询在【独立进程】跑(WeWorkFinanceSdk 是 Go c-shared 库, 嵌进 app-server 会 segfault
把主进程带崩, 见 scripts/wx_finance_poller.py)。poller 拉到美团卡片/截图后 POST 到这里, 由
app-server 打比价信号 —— set_pending 在 app-server 进程内存, 心跳才 pop 得到, 故必须回到本进程。
靠共享密钥头 X-Internal-Secret(== settings.INTERNAL_API_SECRET)校验;未配置 → 503。
"""
from __future__ import annotations
import hmac
import logging
from typing import Annotated
from fastapi import APIRouter, Header, HTTPException, status
from app.core import wx_poc_signal
from app.core.config import settings
from app.schemas.wx_finance import WxFinancePendingIn, WxFinancePendingOut
logger = logging.getLogger("shagua.internal.wx_finance")
router = APIRouter(prefix="/internal", tags=["internal"])
def _check_secret(x_internal_secret: str | None) -> None:
"""共享密钥校验。未配置 → 503(挡裸奔);不匹配 → 401(常量时间比较)。"""
configured = settings.INTERNAL_API_SECRET
if not configured:
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail="internal api not configured",
)
if not x_internal_secret or not hmac.compare_digest(x_internal_secret, configured):
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED, detail="invalid internal secret"
)
@router.post(
"/wx-finance/pending",
response_model=WxFinancePendingOut,
summary="会话存档 poller 触发比价信号(独立进程→app-server, 打 set_pending)",
)
def wx_finance_pending(
payload: WxFinancePendingIn,
x_internal_secret: Annotated[str | None, Header()] = None,
) -> WxFinancePendingOut:
_check_secret(x_internal_secret)
device_id = settings.WX_POC_TEST_DEVICE_ID
if not device_id:
logger.warning("wx_finance pending: WX_POC_TEST_DEVICE_ID 未配置, 忽略信号")
return WxFinancePendingOut(ok=False)
wx_poc_signal.set_pending(device_id, payload.source)
logger.info(
"wx_finance PoC: 已给测试设备 %s 打比价信号 source=%s (触发=%s)",
device_id, payload.source, payload.kind,
)
return WxFinancePendingOut(ok=True)
+299 -16
View File
@@ -12,19 +12,36 @@ from __future__ import annotations
import logging
from fastapi import APIRouter, HTTPException, Request, status
from fastapi import APIRouter, HTTPException, Request
from sqlalchemy.exc import IntegrityError
from app.api.deps import CurrentUser, DbSession
from app.core import test_account
from app.core.ratelimit import enforce_rate_limit
from app.core.security import TokenError, decode_token, issue_token_pair
from app.core.ratelimit import (
RateLimitRule,
check_rate_limits,
enforce_rate_limit,
record_rate_limits,
)
from app.core.security import (
TokenError,
create_bind_ticket,
create_conflict_ticket,
decode_bind_ticket,
decode_conflict_ticket,
decode_token,
issue_token_pair,
)
from app.integrations import wxpay
from app.integrations.jiguang import JiguangError, mask_phone, verify_and_get_phone
from app.integrations.sms import SmsError, send_code, verify_code
from app.repositories import onboarding as onboarding_repo
from app.repositories import phone_rebind as rebind_repo
from app.repositories import user as user_repo
from app.schemas.auth import (
JverifyLoginRequest,
LogoutResponse,
OccupiedAccountInfo,
RefreshRequest,
SmsLoginRequest,
SmsSendRequest,
@@ -32,6 +49,13 @@ from app.schemas.auth import (
TokenPair,
TokenWithUser,
UserOut,
WechatBindPhoneJverifyRequest,
WechatBindPhoneSmsRequest,
WechatBindResultResponse,
WechatConflictContinueRequest,
WechatConflictRebindRequest,
WechatLoginRequest,
WechatLoginResponse,
)
logger = logging.getLogger("shagua.auth")
@@ -40,9 +64,10 @@ router = APIRouter(prefix="/api/v1/auth", tags=["auth"])
# 手机号登录防刷:同一设备(device_id) + 同一 IP 每小时最多的登录尝试次数(成功/失败都计)。
SMS_LOGIN_MAX_PER_HOUR = 5
# 发码防刷:同一设备(device_id) + 同一 IP 每小时最多的发码次数。
# 发码防刷(同一设备 device_id + 同一 IP,**只按成功发码计数**;被单号 60s 冷却挡下的重发不占额度):
# 堵「换手机号绕开单号 60s 冷却」的洞 —— 冷却是单号维度,一机换号能绕开。
SMS_SEND_MAX_PER_HOUR_PER_DEVICE = 5
SMS_SEND_MAX_PER_HOUR_PER_DEVICE = 5 # 每小时上限
SMS_SEND_MAX_PER_DAY_PER_DEVICE = 20 # 每天上限(再叠一层日封顶,挡低频长时间轰炸)
def _login_response(
@@ -99,23 +124,26 @@ def sms_send(req: SmsSendRequest, request: Request) -> SmsSendResponse:
logger.info("test_account sms_send short-circuit (不真发)")
return SmsSendResponse(sent=True, mock=True, cooldown_sec=0)
# 防刷:同一设备(device_id) + 同一 IP 每小时最多 SMS_SEND_MAX_PER_HOUR_PER_DEVICE 次发码
# 补「换手机号绕开单号 60s 冷却」的洞(冷却是单号维度,一机换号能绕);设备维度按机器封顶,
# 挡短信轰炸/烧钱。放在真发(send_code)之前 → 超限直接拦下、不真发短信。
enforce_rate_limit(
request,
scope="sms-send-device",
subject=req.device_id,
limit=SMS_SEND_MAX_PER_HOUR_PER_DEVICE,
window_sec=3600,
detail="操作过于频繁,请稍后再试",
)
# 发码防刷:同一设备(device_id) + 同一 IP,每小时 / 每天两道闸,**均只按成功发码计数**
# 补「换手机号绕开单号 60s 冷却」的洞(冷却是单号维度,一机换号能绕);设备维度按机器封顶,挡短信轰炸/烧钱。
# 关键:被单号 60s 冷却挡下的重发是「没真发、没烧钱」→ 不该占额度。故 check(先判)放在真发之前
# (超限直接 429、不真发),record(计数)只在 send_code 成功后调 —— 冷却/供应商失败抛 429 时直接返回、不计数。
send_rules = [
RateLimitRule("sms-send-device", SMS_SEND_MAX_PER_HOUR_PER_DEVICE, 3600,
"操作过于频繁,请稍后再试"),
RateLimitRule("sms-send-device-daily", SMS_SEND_MAX_PER_DAY_PER_DEVICE, 86400,
"今日验证码发送次数过多,请明天再试"),
]
check_rate_limits(request, subject=req.device_id, rules=send_rules)
try:
cooldown = send_code(req.phone)
except SmsError as e:
raise HTTPException(status_code=e.status_code, detail=str(e)) from e
# 发码成功 → 两道闸各 +1(被单号冷却挡下的重发走不到这里,故不占额度)
record_rate_limits(request, subject=req.device_id, rules=send_rules)
from app.core.config import settings # 局部 import 避免循环
return SmsSendResponse(sent=True, mock=settings.SMS_MOCK, cooldown_sec=cooldown)
@@ -166,6 +194,261 @@ def sms_login(req: SmsLoginRequest, request: Request, db: DbSession) -> TokenWit
return _login_response(user, onboarding_completed=completed)
# ===================== 微信登录 =====================
@router.post(
"/wechat-login",
response_model=WechatLoginResponse,
summary="微信登录(openid 命中即登入,否则发绑号令牌)",
)
def wechat_login(req: WechatLoginRequest, db: DbSession) -> WechatLoginResponse:
from app.core.config import settings # 局部 import,避免循环
# 微信登录只需 code→openid(sns/oauth2),不需要商户转账证书;故只校验 APP_ID/SECRET。
if not (settings.WECHAT_APP_ID and settings.WECHAT_APP_SECRET):
raise HTTPException(status_code=503, detail="wechat login not configured")
try:
info = wxpay.code_to_userinfo(req.code) # {openid, nickname, avatar_url, raw};失败抛 ValueError
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e)) from e
openid = info["openid"]
user = user_repo.get_user_by_wechat_openid(db, openid)
if user is not None:
# openid 命中 → 直接登入(绝不套用提现 bind-wechat 的"撞号即 409"逻辑)
if user.status != "active":
raise HTTPException(status_code=403, detail="account disabled")
user_repo.touch_last_login(db, user)
completed = onboarding_repo.is_completed(db, user_id=user.id, device_id=req.device_id)
logger.info("wechat_login hit user_id=%d openid=%s*** onboarded=%s", user.id, openid[:6], completed)
return WechatLoginResponse(
status="logged_in",
token=_login_response(user, onboarding_completed=completed),
)
# 未命中 → 签发短时 bind_ticket,进手机号绑定流程(账号此刻还不建)
ticket = create_bind_ticket(
openid=openid,
wechat_nickname=info["nickname"],
wechat_avatar_url=info["avatar_url"],
)
logger.info("wechat_login new openid=%s*** issue bind_ticket", openid[:6])
return WechatLoginResponse(
status="need_bind_phone",
bind_ticket=ticket,
wechat_nickname=info["nickname"],
wechat_avatar_url=info["avatar_url"],
)
def _finish_wechat_bind(
db,
*,
openid: str,
wechat_nickname: str | None,
wechat_avatar_url: str | None,
phone: str,
device_id: str,
) -> WechatBindResultResponse:
"""绑手机建号的公共尾段:手机号被占用 → 返回 phone_occupied(M2 处理 3 选 1);
未占用 → 新建微信账号(channel=wechat,昵称头像取微信)→ 签 token 登入。"""
existing = user_repo.get_user_by_phone(db, phone)
if existing is not None:
from app.core.config import settings # 局部 import,避免循环
ticket = create_conflict_ticket(
openid=openid,
wechat_nickname=wechat_nickname,
wechat_avatar_url=wechat_avatar_url,
phone=phone,
)
blocked = rebind_repo.rebound_within_days(db, phone, settings.PHONE_REBIND_LIMIT_DAYS)
logger.info(
"wechat bind phone occupied phone=%s by user_id=%d has_wechat=%s",
mask_phone(phone), existing.id, bool(existing.wechat_openid),
)
return WechatBindResultResponse(
status="phone_occupied",
occupied_account=OccupiedAccountInfo(
nickname=existing.nickname,
avatar_url=existing.avatar_url,
created_at=existing.created_at,
has_wechat=bool(existing.wechat_openid),
),
conflict_ticket=ticket,
rebind_available=not blocked,
rebind_blocked_days=(
rebind_repo.remaining_block_days(db, phone, settings.PHONE_REBIND_LIMIT_DAYS)
if blocked else 0
),
)
user = user_repo.create_wechat_user(
db,
phone=phone,
openid=openid,
wechat_nickname=wechat_nickname,
wechat_avatar_url=wechat_avatar_url,
)
completed = onboarding_repo.is_completed(db, user_id=user.id, device_id=device_id)
logger.info("wechat bind ok user_id=%d phone=%s openid=%s*** onboarded=%s",
user.id, mask_phone(phone), openid[:6], completed)
return WechatBindResultResponse(
status="logged_in",
token=_login_response(user, onboarding_completed=completed),
)
@router.post(
"/wechat/bind-phone/sms",
response_model=WechatBindResultResponse,
summary="微信登录·其他手机号(短信)绑定",
)
def wechat_bind_phone_sms(
req: WechatBindPhoneSmsRequest, request: Request, db: DbSession
) -> WechatBindResultResponse:
try:
claims = decode_bind_ticket(req.bind_ticket)
except TokenError as e:
raise HTTPException(status_code=401, detail="授权已过期,请重新用微信登录") from e
# 防刷:同 sms/login,按 设备+IP 每小时限流(放在验证码校验之前,失败也计数)
enforce_rate_limit(
request,
scope="wechat-bind-sms-device",
subject=req.device_id,
limit=SMS_LOGIN_MAX_PER_HOUR,
window_sec=3600,
detail="登录尝试过于频繁,请稍后再试",
)
if not verify_code(req.phone, req.code):
raise HTTPException(status_code=400, detail="invalid sms code")
return _finish_wechat_bind(
db,
openid=claims["openid"],
wechat_nickname=claims["wnk"],
wechat_avatar_url=claims["wav"],
phone=req.phone,
device_id=req.device_id,
)
@router.post(
"/wechat/bind-phone/jverify",
response_model=WechatBindResultResponse,
summary="微信登录·本机号(极光)绑定",
)
def wechat_bind_phone_jverify(
req: WechatBindPhoneJverifyRequest, db: DbSession
) -> WechatBindResultResponse:
try:
claims = decode_bind_ticket(req.bind_ticket)
except TokenError as e:
raise HTTPException(status_code=401, detail="授权已过期,请重新用微信登录") from e
try:
phone = verify_and_get_phone(req.login_token)
except JiguangError as e:
logger.error("[JG] verify+decrypt failed: %s", e, exc_info=True)
raise HTTPException(status_code=502, detail=f"jiguang verify failed: {e}") from e
return _finish_wechat_bind(
db,
openid=claims["openid"],
wechat_nickname=claims["wnk"],
wechat_avatar_url=claims["wav"],
phone=phone,
device_id=req.device_id,
)
# ===================== 微信占用冲突(M2) =====================
@router.post(
"/wechat/conflict/continue",
response_model=WechatBindResultResponse,
summary="微信占用冲突·继续绑定(登录老账号,能绑就绑)",
)
def wechat_conflict_continue(
req: WechatConflictContinueRequest, request: Request, db: DbSession
) -> WechatBindResultResponse:
try:
claims = decode_conflict_ticket(req.conflict_ticket)
except TokenError as e:
raise HTTPException(status_code=401, detail="操作超时,请重新用微信登录") from e
enforce_rate_limit(
request, scope="wechat-conflict-device", subject=req.device_id,
limit=SMS_LOGIN_MAX_PER_HOUR, window_sec=3600, detail="操作过于频繁,请稍后再试",
)
user = user_repo.get_user_by_phone(db, claims["phone"])
if user is None:
# P 期间被腾空(老账号改号/注销)→ 前提已变,让前端重走
raise HTTPException(status_code=409, detail="账号状态已变化,请重新登录")
if user.status != "active":
raise HTTPException(status_code=403, detail="account disabled")
if user.wechat_openid is None:
try:
user_repo.attach_wechat_to_user(
db, user, openid=claims["openid"],
wechat_nickname=claims["wnk"], wechat_avatar_url=claims["wav"],
)
except IntegrityError:
db.rollback() # openid 被别处绑走 → 只登入不绑
user_repo.touch_last_login(db, user)
else:
user_repo.touch_last_login(db, user) # X 已绑别的微信 → 只登入,丢弃本次 openid
completed = onboarding_repo.is_completed(db, user_id=user.id, device_id=req.device_id)
logger.info("wechat conflict continue user_id=%d openid=%s***", user.id, claims["openid"][:6])
return WechatBindResultResponse(
status="logged_in",
token=_login_response(user, onboarding_completed=completed),
)
@router.post(
"/wechat/conflict/rebind",
response_model=WechatBindResultResponse,
summary="微信占用冲突·换绑(注销老账号+用该号重建全新账号)",
)
def wechat_conflict_rebind(
req: WechatConflictRebindRequest, request: Request, db: DbSession
) -> WechatBindResultResponse:
from app.core.config import settings # 局部 import,避免循环
try:
claims = decode_conflict_ticket(req.conflict_ticket)
except TokenError as e:
raise HTTPException(status_code=401, detail="操作超时,请重新用微信登录") from e
enforce_rate_limit(
request, scope="wechat-conflict-device", subject=req.device_id,
limit=SMS_LOGIN_MAX_PER_HOUR, window_sec=3600, detail="操作过于频繁,请稍后再试",
)
phone = claims["phone"]
if rebind_repo.rebound_within_days(db, phone, settings.PHONE_REBIND_LIMIT_DAYS):
days = rebind_repo.remaining_block_days(db, phone, settings.PHONE_REBIND_LIMIT_DAYS)
raise HTTPException(status_code=409, detail=f"该手机号 {days} 天内已换绑过,暂不能再次换绑")
user = user_repo.rebind_account(
db, phone=phone, openid=claims["openid"],
wechat_nickname=claims["wnk"], wechat_avatar_url=claims["wav"],
)
completed = onboarding_repo.is_completed(db, user_id=user.id, device_id=req.device_id)
logger.info("wechat conflict rebind new_user_id=%d phone=%s openid=%s***",
user.id, mask_phone(phone), claims["openid"][:6])
return WechatBindResultResponse(
status="logged_in",
token=_login_response(user, onboarding_completed=completed),
)
# ===================== Refresh =====================
@router.post("/refresh", response_model=TokenPair, summary="用 refresh_token 换新 token 对")
+3 -22
View File
@@ -14,28 +14,18 @@ import logging
from fastapi import APIRouter
from app.api.deps import CurrentUser, DbSession
from app.core import wx_poc_signal
from app.repositories import device as device_repo
from app.schemas.device import (
DeviceOut,
DeviceRegisterRequest,
HeartbeatRequest,
HeartbeatResponse,
LivenessAckRequest,
LivenessOut,
OkResponse,
PendingCompare,
)
logger = logging.getLogger("shagua.device")
# PoC:源平台代号 → Android 包名(前端 launch 用)。poller 按分享的小程序卡区分平台。
# 客户端只认 source_package:用它反查平台名(弹窗"原平台")+ launch 对应 App,故新增平台只需在此登记包名。
_SOURCE_PACKAGES = {
"meituan": "com.sankuai.meituan",
"jd": "com.jingdong.app.mall", # 京东主 App(含京东秒送/外卖),后端 intent skill=jd_waimai
}
router = APIRouter(prefix="/api/v1/device", tags=["device"])
@@ -62,12 +52,12 @@ def register_device(
return DeviceOut.model_validate(device)
@router.post("/heartbeat", response_model=HeartbeatResponse, summary="上报心跳")
@router.post("/heartbeat", response_model=OkResponse, summary="上报心跳")
def report_heartbeat(
req: HeartbeatRequest,
user: CurrentUser,
db: DbSession,
) -> HeartbeatResponse:
) -> OkResponse:
device_repo.touch_heartbeat(
db,
user_id=user.id,
@@ -75,16 +65,7 @@ def report_heartbeat(
accessibility_enabled=req.accessibility_enabled,
registration_id=req.registration_id,
)
# PoC:该设备有"待比价"信号则带回(只对写死的测试设备生效, 取走即清、只弹一次)
source = wx_poc_signal.pop_pending(req.device_id)
pending = None
if source:
pending = PendingCompare(
source_platform=source,
source_package=_SOURCE_PACKAGES.get(source, ""),
)
logger.info("heartbeat 下发比价信号 device=%s source=%s", req.device_id, source)
return HeartbeatResponse(pending_compare=pending)
return OkResponse()
@router.get("/liveness", response_model=LivenessOut, summary="查询本机掉线告警(后置检测)")
-159
View File
@@ -1,159 +0,0 @@
"""微信客服(企业微信)消息接收回调 /wx/kf/callback(裸路径, 非 /api/v1)。
对应企业微信「微信客服」→ 客服账号 → 接收消息回调里填的 URL。是服务号版(wx_mp.py)的
平行实现:下游(set_pending → 心跳下发 → 前端弹窗)完全一致, 只有上游"如何拿到用户消息"不同 ——
服务号是微信把消息直接 POST 给你;微信客服是 POST 一个事件通知, 你再用 <Token> 主动 sync_msg 拉。
GET : URL 接入验证 → msg_signature 验签(含密文 echostr) → 解密 echostr → 返回明文。
⚠️ 与公众号明文模式不同:企业微信 echostr 是密文, 必须解密后返回。
POST : 收事件(密文) → msg_signature 验签 → AES 解密 → 解析 XML。
Event=kf_msg_or_event → 取 <Token>/<OpenKfId> → sync_msg 增量拉消息 →
客户(origin=3)发来的图片(image) → set_pending(测试设备, "meituan") + 下载落盘。
其余消息/事件记日志忽略。一律返回 "success"(异常也吞掉, 不让企业微信重试轰炸)。
回调加密与公众号「安全模式」同源, 直接复用 integrations/wx_mp_crypto(验签 + AES-256-CBC 解密),
差别只在 receiveid 用 corpid(公众号是 appid)。MVP 只做"收图打信号 + 落盘证明链路通"
"""
from __future__ import annotations
import logging
import xml.etree.ElementTree as ET
from pathlib import Path
from typing import Any
from fastapi import APIRouter, Request
from fastapi.responses import PlainTextResponse
from app.core import wx_poc_signal
from app.core.config import settings
from app.integrations import wx_kf_client, wx_mp_crypto
logger = logging.getLogger("shagua.wx_kf")
router = APIRouter(prefix="/wx/kf", tags=["wx-kf"])
# 收到的截图暂存目录(MVP 验证用;接上识别后可改为不落盘或定期清理)
_INBOX = Path(settings.MEDIA_ROOT) / "wx_kf_inbox"
@router.get("/callback", include_in_schema=False)
async def verify(
msg_signature: str = "", timestamp: str = "", nonce: str = "", echostr: str = ""
) -> PlainTextResponse:
"""企业微信 URL 接入验证:验签(密文 echostr 参与)→ 解密 echostr → 返回明文。"""
if not settings.wx_kf_callback_configured:
logger.warning("wx_kf verify: 回调凭证未配齐(CorpId/Token/AESKey/Secret)")
return PlainTextResponse("", status_code=503)
if not wx_mp_crypto.verify_msg_signature(
settings.WX_KF_TOKEN, timestamp, nonce, echostr, msg_signature
):
logger.warning("wx_kf verify: signature 校验失败 ts=%s nonce=%s", timestamp, nonce)
return PlainTextResponse("invalid signature", status_code=403)
try:
plain = wx_mp_crypto.decrypt_message(
settings.WX_KF_AES_KEY, settings.WX_KF_CORP_ID, echostr
)
except Exception:
logger.exception("wx_kf verify: echostr 解密失败")
return PlainTextResponse("decrypt failed", status_code=403)
return PlainTextResponse(plain)
@router.post("/callback", include_in_schema=False)
async def receive(request: Request) -> PlainTextResponse:
"""收事件通知(安全模式)。任何异常都吞掉返回 success, 不让企业微信重试轰炸, 靠日志排查。"""
if not settings.wx_kf_callback_configured:
logger.warning("wx_kf receive: 回调凭证未配齐")
return PlainTextResponse("success")
try:
body = (await request.body()).decode("utf-8")
qp = request.query_params
# 安全模式外层 XML 只有 ToUserName + Encrypt。来源=企业微信服务器(HTTPS)+ 下面验签,
# 且 stdlib ET 不扩展外部实体, XXE 不适用。
encrypt = ET.fromstring(body).findtext("Encrypt") or ""
if not wx_mp_crypto.verify_msg_signature(
settings.WX_KF_TOKEN,
qp.get("timestamp", ""),
qp.get("nonce", ""),
encrypt,
qp.get("msg_signature", ""),
):
logger.warning("wx_kf receive: msg_signature 校验失败")
return PlainTextResponse("success")
xml = wx_mp_crypto.decrypt_message(
settings.WX_KF_AES_KEY, settings.WX_KF_CORP_ID, encrypt
)
await _handle_event(ET.fromstring(xml))
except Exception:
logger.exception("wx_kf receive: 处理异常")
return PlainTextResponse("success")
async def _handle_event(root: ET.Element) -> None:
"""客服事件:MsgType=event 且 Event=kf_msg_or_event → 用 Token 增量拉消息。"""
event = root.findtext("Event") or ""
if event != "kf_msg_or_event":
logger.info("wx_kf 收到事件(暂忽略): event=%s", event)
return
token = root.findtext("Token") or ""
open_kfid = root.findtext("OpenKfId") or ""
msgs = await wx_kf_client.sync_messages(token, open_kfid)
logger.info("wx_kf sync_msg 拉到 %d 条 open_kfid=%s", len(msgs), open_kfid)
for msg in msgs:
await _handle_message(msg)
async def _handle_message(msg: dict[str, Any]) -> None:
"""处理单条客服消息。只关心【客户发来的】(origin=3)图片 / 小程序卡片:打比价信号(+ 落盘)。
origin:3=客户发的 / 4=系统推送 / 5=接待人员发的。接待人员或系统消息一律不触发比价。
两种触发物对应场景:image=用户发结算页截图;miniprogram=用户分享美团小程序卡片(见需求截图)。
"""
if msg.get("origin") != 3:
return
msgtype = msg.get("msgtype") or ""
external_userid = msg.get("external_userid") or ""
if msgtype == "image":
media_id = (msg.get("image") or {}).get("media_id", "")
logger.info(
"wx_kf 收到图片: external_userid=%s media_id=%s", external_userid, media_id
)
_fire_pending("image")
await _save_media(media_id, external_userid)
elif msgtype == "miniprogram":
mp = msg.get("miniprogram") or {}
logger.info(
"wx_kf 收到小程序卡片: external_userid=%s title=%s appid=%s",
external_userid,
mp.get("title"),
mp.get("appid"),
)
# 卡片无可下载媒体, 只打信号;源平台/订单识别(可据 appid 判平台)留后续, 现写死美团。
_fire_pending("miniprogram")
else:
logger.info(
"wx_kf 收到消息(暂忽略): external_userid=%s type=%s", external_userid, msgtype
)
def _fire_pending(kind: str) -> None:
"""PoC:给写死的测试设备打"从美团比价"信号(下次心跳带回前端弹窗, 与服务号版完全一致)。"""
poc_dev = settings.WX_POC_TEST_DEVICE_ID
if not poc_dev:
return
wx_poc_signal.set_pending(poc_dev, "meituan")
logger.info("wx_kf PoC: 已给测试设备 %s 打比价信号 source=meituan (触发=%s)", poc_dev, kind)
async def _save_media(media_id: str, external_userid: str) -> None:
"""图片消息只给 media_id, 走临时素材下载接口拉回落盘(PoC 验证用)。"""
content = await wx_kf_client.download_media(media_id)
if not content:
return
try:
_INBOX.mkdir(parents=True, exist_ok=True)
dest = _INBOX / f"{external_userid[:16]}_{media_id[:16]}.jpg"
dest.write_bytes(content)
logger.info("wx_kf 图片已落盘: %s (%d bytes)", dest, len(content))
except Exception:
logger.exception("wx_kf 图片落盘失败 media_id=%s", media_id)
-112
View File
@@ -1,112 +0,0 @@
"""微信服务号消息接收回调 /wx/mp/callback(裸路径, 非 /api/v1)。
对应公众平台"设置与开发 → 基本配置 → 服务器配置"里填的 URL。
GET : URL 接入验证 → 校验 signature → 原样返回 echostr(明文)
POST : 收用户消息(安全模式密文) → msg_signature 验签 → AES 解密 → 解析 XML
图片消息(MsgType=image): 打日志(openid/PicUrl/MediaId) + 用 PicUrl 下载落盘
其余消息/事件: 记一条日志忽略。一律返回 "success"(微信据此不再重试)。
MVP 只做"收到图片并落盘"证明链路通;后续接:识别平台+订单 → 打 pending_compare 标记
→ 心跳带回前端弹窗,以及 openid↔device 绑定。任何异常都返回 "success"、不让微信重试轰炸。
"""
from __future__ import annotations
import logging
import xml.etree.ElementTree as ET
from pathlib import Path
import httpx
from fastapi import APIRouter, Request
from fastapi.responses import PlainTextResponse
from app.core import wx_poc_signal
from app.core.config import settings
from app.integrations import wx_mp_crypto
logger = logging.getLogger("shagua.wx_mp")
router = APIRouter(prefix="/wx/mp", tags=["wx-mp"])
# 收到的截图暂存目录(MVP 验证用;接上识别后可改为不落盘或定期清理)
_INBOX = Path(settings.MEDIA_ROOT) / "wx_inbox"
@router.get("/callback", include_in_schema=False)
async def verify(
signature: str = "", timestamp: str = "", nonce: str = "", echostr: str = ""
) -> PlainTextResponse:
"""微信 URL 接入验证:校验 signature 通过则原样返回 echostr。"""
if not settings.WX_MP_TOKEN:
logger.warning("wx_mp verify: WX_MP_TOKEN 未配置")
return PlainTextResponse("", status_code=503)
if wx_mp_crypto.verify_url_signature(
settings.WX_MP_TOKEN, timestamp, nonce, signature
):
return PlainTextResponse(echostr)
logger.warning("wx_mp verify: signature 校验失败 ts=%s nonce=%s", timestamp, nonce)
return PlainTextResponse("invalid signature", status_code=403)
@router.post("/callback", include_in_schema=False)
async def receive(request: Request) -> PlainTextResponse:
"""收用户消息(安全模式)。任何异常都吞掉返回 success,不让微信重试轰炸,靠日志排查。"""
if not settings.wx_mp_callback_configured:
logger.warning("wx_mp receive: 回调凭证未配齐(Token/AESKey/AppID)")
return PlainTextResponse("success")
try:
body = (await request.body()).decode("utf-8")
qp = request.query_params
# 安全模式外层 XML 只有 ToUserName + Encrypt。来源=微信服务器(HTTPS)+ 下面验签,
# 且 stdlib ET 不扩展外部实体, XXE 不适用。
encrypt = ET.fromstring(body).findtext("Encrypt") or ""
if not wx_mp_crypto.verify_msg_signature(
settings.WX_MP_TOKEN,
qp.get("timestamp", ""),
qp.get("nonce", ""),
encrypt,
qp.get("msg_signature", ""),
):
logger.warning("wx_mp receive: msg_signature 校验失败")
return PlainTextResponse("success")
xml = wx_mp_crypto.decrypt_message(
settings.WX_MP_AES_KEY, settings.WX_MP_APPID, encrypt
)
await _handle_message(ET.fromstring(xml))
except Exception:
logger.exception("wx_mp receive: 处理异常")
return PlainTextResponse("success")
async def _handle_message(root: ET.Element) -> None:
openid = root.findtext("FromUserName") or ""
msg_type = root.findtext("MsgType") or ""
if msg_type == "image":
pic_url = root.findtext("PicUrl") or ""
media_id = root.findtext("MediaId") or ""
logger.info(
"wx_mp 收到图片: openid=%s media_id=%s pic_url=%s", openid, media_id, pic_url
)
# PoC: 写死"该测试设备要从美团比价", 下次心跳带回前端弹选平台窗
poc_dev = settings.WX_POC_TEST_DEVICE_ID
if poc_dev:
wx_poc_signal.set_pending(poc_dev, "meituan")
logger.info("wx_mp PoC: 已给测试设备 %s 打比价信号 source=meituan", poc_dev)
await _download(pic_url, openid, media_id)
else:
logger.info("wx_mp 收到消息(暂忽略): openid=%s type=%s", openid, msg_type)
async def _download(pic_url: str, openid: str, media_id: str) -> None:
"""用 PicUrl 直接下载图片落盘。PicUrl 是临时公网链接,无需 access_token / IP 白名单。"""
if not pic_url:
return
try:
_INBOX.mkdir(parents=True, exist_ok=True)
async with httpx.AsyncClient(timeout=15) as client:
resp = await client.get(pic_url)
resp.raise_for_status()
dest = _INBOX / f"{openid[:12]}_{media_id[:16]}.jpg"
dest.write_bytes(resp.content)
logger.info("wx_mp 图片已落盘: %s (%d bytes)", dest, len(resp.content))
except Exception:
logger.exception("wx_mp 图片下载失败 pic_url=%s", pic_url)
+56 -54
View File
@@ -44,6 +44,11 @@ class Settings(BaseSettings):
JWT_ALGORITHM: str = "HS256"
JWT_ACCESS_TOKEN_EXPIRE_MINUTES: int = 120
JWT_REFRESH_TOKEN_EXPIRE_DAYS: int = 30
# 微信登录未命中 openid 时签发的"待绑手机"令牌有效期(JWT_SECRET_KEY 签名,typ=wechat_bind;
# 见 security.create_bind_ticket)。需覆盖"授权→输手机号→收短信→输验证码"整个绑定流程。
WECHAT_BIND_TICKET_EXPIRE_MINUTES: int = 10
# 一个手机号 30 天内最多换绑一次(微信占用冲突页的"换绑"动作)。见 phone_rebind_log。
PHONE_REBIND_LIMIT_DAYS: int = 30
# ===== Admin 后台 =====
# admin 用独立 JWT secret(≠ JWT_SECRET_KEY),App 用户 token 无法越权访问后台。
@@ -81,6 +86,7 @@ class Settings(BaseSettings):
SMS_SIGN_ID: int = 31729 # 极光短信签名 ID(非机密,可被 .env 覆盖)
SMS_TEMPLATE_ID: int = 1 # 极光短信模板 ID(变量名 code,有效期 5 分钟)
SMS_CODE_LENGTH: int = 6 # 验证码位数(本服务生成;前端 code 字段 4-8 位兼容)
SMS_DAILY_LIMIT_PER_PHONE: int = 10 # 单手机号每日发送上限(防刷 + 控费)
SMS_MAX_VERIFY_ATTEMPTS: int = 5 # 单个验证码最多校验失败次数,超过即作废(防爆破)
# ===== 测试账号(release 包全流程联调用)=====
@@ -106,6 +112,9 @@ class Settings(BaseSettings):
# 美团调用走的代理。本机开发直连美团会 SSL EOF,需填 http://127.0.0.1:7897;
# 线上国内服务器留空(=直连)。见 .env.example 与 integrations/meituan.py。
MT_CPS_PROXY: str = ""
# 本地开发:开启后 /feed 接口直接返回 mock 数据,不调美团 API、不查离线库,
# 方便前端联调 feed 卡片样式、分页、距离排序等 UI。生产必须 false。
MT_CPS_MOCK_FEED: bool = True
@property
def mt_cps_configured(self) -> bool:
@@ -136,40 +145,6 @@ class Settings(BaseSettings):
# (不跳授权、不拿 openid),整套微信代码保留。审核通过后 .env 置 true + 重启即启用,无需改代码。
WX_MP_OAUTH_ENABLED: bool = False
# ===== 微信服务号(消息接收回调) =====
# 用户发消息给服务号 → 微信 POST 到 /wx/mp/callback(安全模式:Token 验签 + AESKey 解密)。
# 区别于上面的网页授权(那是落地页拿 openid);这里是被动收用户主动发来的消息(截图比价入口)。
WX_MP_TOKEN: str = "" # 公众平台"服务器配置"的 Token(URL 验签 + 消息验签)
WX_MP_AES_KEY: str = "" # EncodingAESKey(43 位, 消息 AES-256-CBC 加解密)
# ===== 微信截图比价 PoC(临时验证链路, 验证后删) =====
# 只对这台写死的测试设备下发"从美团比价"信号;空=不触发任何设备(部署上线零风险)。
# 收图 → 给此 device_id 打 pending → 该设备下次心跳带回 → 前端弹选平台窗。
WX_POC_TEST_DEVICE_ID: str = ""
# ===== 微信客服(企业微信;截图比价上游入口 —— 服务号版的平行实现) =====
# 用户在「微信客服」会话发消息 → 企业微信 POST 事件到 /wx/kf/callback → 后端 sync_msg 拉消息。
# 回调加密与公众号「安全模式」同源(复用 wx_mp_crypto), 但 receiveid 用 corpid 且 echostr 需解密。
# 拉消息 / 下载图片走 qyapi, 需 access_token(corpid + secret 换取)。全空=回调端点拒收, 零风险。
WX_KF_CORP_ID: str = "" # 企业微信 corpid(回调解密 receiveid + 换 access_token)
WX_KF_SECRET: str = "" # 「微信客服」Secret(换 access_token)
WX_KF_TOKEN: str = "" # 客服「接收消息」回调的 Token(URL 验签 + 消息验签)
WX_KF_AES_KEY: str = "" # 客服「接收消息」回调的 EncodingAESKey(AES-256-CBC 解密)
# ===== 微信会话内容存档(企业微信;"发给成员"那条路的截图/美团卡片比价上游入口) =====
# 用户把美团小程序卡片/截图发给企业微信【成员】(非客服) → 会话存档 GetChatData 轮询拉取 →
# RSA+AES 解密 → 识别 weapp/image → 给测试设备打比价信号。下游(心跳→弹窗→比价)复用。
# 需后台开通「会话内容存档」+ 上传 RSA 公钥 + 配可信IP + 拿存档 Secret;.so 放服务器。
# corpid 复用上面的 WX_KF_CORP_ID(同一企业)。ENABLED 默认关 → 未配好前零影响。
WX_FINANCE_ENABLED: bool = False # 总开关(worker 是否启动)
WX_FINANCE_SECRET: str = "" # 会话存档【专用】Secret(≠微信客服/自建应用)
WX_FINANCE_SDK_PATH: str = "./libWeWorkFinanceSdk_C.so" # WeWorkFinanceSdk C 库(Linux .so)
WX_FINANCE_PRIVATE_KEY_PATH: str = "./secrets/wx_finance_private.pem" # RSA 私钥(解 encrypt_random_key)
WX_FINANCE_POLL_INTERVAL_SEC: float = 0.5 # GetChatData 轮询间隔秒(无回调、自己定频;支持亚秒如 0.5)
WX_FINANCE_SEQ_FILE: str = "./data/wx_finance_seq.txt" # seq 游标持久化(防丢/重复)
WX_FINANCE_RECEIVER_USERID: str = "" # 接收成员 userid(万朗杰);只处理【非他发】的消息, 空=不过滤
WX_FINANCE_INTERNAL_URL: str = "http://127.0.0.1:8770" # poller 通知 app-server 打信号的内部地址(同机)
@property
def wx_mp_configured(self) -> bool:
"""服务号网页授权凭证齐全(缺则落地页不发起授权,降级为无 openid)。"""
@@ -180,26 +155,6 @@ class Settings(BaseSettings):
"""落地页是否真正发起微信授权 = 凭证齐全 且 总开关开。"""
return self.wx_mp_configured and self.WX_MP_OAUTH_ENABLED
@property
def wx_mp_callback_configured(self) -> bool:
"""消息接收回调凭证齐全(Token + AESKey + AppID)。缺则回调端点拒绝处理消息。"""
return bool(self.WX_MP_TOKEN and self.WX_MP_AES_KEY and self.WX_MP_APPID)
@property
def wx_kf_callback_configured(self) -> bool:
"""微信客服回调凭证齐全(CorpId + Token + AESKey + Secret)。缺则回调端点拒绝处理。"""
return bool(
self.WX_KF_CORP_ID
and self.WX_KF_TOKEN
and self.WX_KF_AES_KEY
and self.WX_KF_SECRET
)
@property
def wx_finance_configured(self) -> bool:
"""会话存档 worker 是否该启动 = 总开关开 且 corpid + 存档 Secret 齐全。"""
return bool(self.WX_FINANCE_ENABLED and self.WX_KF_CORP_ID and self.WX_FINANCE_SECRET)
# ===== 微信支付(商家转账到零钱 / 提现)=====
# 真实凭证放 .env(已 gitignore),证书 .pem 放 secrets/。WECHAT_APP_ID 同时用于
# 微信登录(code 换 openid)与转账,必须与 App 端开放平台 appid 一致。
@@ -223,6 +178,15 @@ class Settings(BaseSettings):
# 进程内自动兑换 worker 的检查间隔(秒):每隔这么久醒一次,跨过北京 0 点就跑一轮。
# 默认 600s=10min,即 0 点后最多 10 分钟内兑完(客户端文案已注明「可能存在延迟」)。
AUTO_EXCHANGE_CHECK_INTERVAL_SEC: int = 600
# === 15 天不活跃清零(app.core.inactivity_reset_worker,worker 常驻)===
# ENABLED 只决定是否**真清**:false(默认)= 只记审计名单、不动钱(dry-run,灰度看名单);
# true = 真清金币 + 折算现金(邀请金不清)。看准名单后再置 true。
INACTIVITY_RESET_ENABLED: bool = False
INACTIVITY_RESET_DAYS: int = 15 # 不活跃阈值(天),第 (N+1) 日 0 点清
INACTIVITY_WARN_DAYS_BEFORE: str = "7,2" # 清零前几天各推一次;""=不推。逗号分隔
INACTIVITY_RESET_RUN_HOUR: int = 3 # 北京时间每日执行点(0-23)
INACTIVITY_NOTIFY_CHANNEL: str = "log" # log(占位) / jpush / sms
INACTIVITY_RESET_CHECK_INTERVAL_SEC: int = 1800 # worker 唤醒间隔(秒)
# 免确认收款授权(用户授权免确认模式)的授权结果回调地址,必须公网可访问 HTTPS、不带参数。
# 发起授权 / 首单顺带授权时作为 authorization_notify_url 传给微信。一期不处理回调内容
# (授权状态靠 query 查询兜底),但微信要求该字段非空,故启用免确认前必须配置;留空时免确认相关接口返回未配置。
@@ -243,6 +207,19 @@ class Settings(BaseSettings):
"""免确认收款授权可用 = 微信支付凭证齐全 + 授权回调地址已配。"""
return bool(self.wxpay_configured and self.WXPAY_AUTH_NOTIFY_URL)
@property
def inactivity_warn_stages(self) -> list[int]:
"""解析 INACTIVITY_WARN_DAYS_BEFORE → 降序去重的提前天数列表。
丢弃非数字 / <=0 / >=RESET_DAYS 的项(空串 → 空列表 = 不推)。"""
out: list[int] = []
for part in (self.INACTIVITY_WARN_DAYS_BEFORE or "").split(","):
part = part.strip()
if part.isdigit():
v = int(part)
if 0 < v < self.INACTIVITY_RESET_DAYS and v not in out:
out.append(v)
return sorted(out, reverse=True)
# ===== 穿山甲激励视频(服务端发奖回调)=====
# 看完激励视频后穿山甲服务器回调本服务发金币(S2S,客户端被破解也刷不到)。
# 穿山甲后台配置的"奖励校验密钥"(m-key),验签用。每个 GroMore 广告位 m-key 不同(后台各自
@@ -370,6 +347,31 @@ class Settings(BaseSettings):
return []
return [o.strip() for o in self.CORS_ALLOW_ORIGINS.split(",") if o.strip()]
# ===== 可观测(OpenObserve 接口指标)=====
# 采集每个接口的 QPS + 耗时 + 错误率,批量直采到 OpenObserve(本地 Docker)。
# 默认关(prod 安全):未开启 → 中间件透传、worker 不启动,整套 no-op。
# 开启需 ENABLED=true 且 ENDPOINT/USER/PASSWORD 齐全(见 observe_configured)。
OBSERVE_ENABLED: bool = False
OBSERVE_ENDPOINT: str = "http://localhost:5080" # OpenObserve base URL
OBSERVE_ORG: str = "default" # 组织名
OBSERVE_STREAM: str = "app_requests" # stream 名(首次上报自动建)
OBSERVE_USER: str = "" # Basic auth 邮箱
OBSERVE_PASSWORD: str = "" # Basic auth 密码/token
OBSERVE_FLUSH_INTERVAL_SEC: float = 5.0 # worker 最长攒批间隔
OBSERVE_BATCH_MAX: int = 200 # 单批最大事件数
OBSERVE_QUEUE_MAX: int = 10000 # 有界队列上限,满则丢
OBSERVE_TIMEOUT_SEC: float = 5.0 # 上报 HTTP 超时
@property
def observe_configured(self) -> bool:
"""观测上报可用 = 总开关开 且 endpoint/账号/密码齐全(缺则整套 no-op)。"""
return bool(
self.OBSERVE_ENABLED
and self.OBSERVE_ENDPOINT
and self.OBSERVE_USER
and self.OBSERVE_PASSWORD
)
@property
def is_prod(self) -> bool:
return self.APP_ENV == "prod"
+145
View File
@@ -0,0 +1,145 @@
"""15 天不活跃清零的进程内每日任务。
仿 daily_exchange_worker:App 启动自带,每 `INACTIVITY_RESET_CHECK_INTERVAL_SEC` 醒一次,
跨进北京新的一天且到达 `INACTIVITY_RESET_RUN_HOUR`(默认 3 点)后跑一轮 `run_once`(预警 + 清零)。
健壮性:
- **逐用户幂等**:清完余额=0 次日不再匹配;预警按 streak 去重。启动补跑 / 多次唤醒 / 重启都安全。
- **同机多进程互斥**:文件锁保证多 worker 只有一个实际跑。
- **常驻 + dry-run 默认**:worker 一直跑;INACTIVITY_RESET_ENABLED=false(默认)只记审计名单、
不动钱(dry-run 灰度看名单),=true 才真清。
⚠️ 这是不可逆批量资金操作(清空金币 + 折算现金,**邀请现金不清**)。口径见
app.repositories.inactivity / app.repositories.activity。
"""
from __future__ import annotations
import asyncio
import contextlib
import logging
import os
import time
from collections.abc import Iterator
from datetime import date, datetime
from pathlib import Path
from sqlalchemy.exc import SQLAlchemyError
from app.core.config import settings
from app.core.rewards import CN_TZ, cn_today
from app.db.session import SessionLocal
from app.integrations.notifier import get_notifier
from app.repositories import inactivity as inactivity_repo
logger = logging.getLogger("shagua.inactivity")
_LOCK_PATH = Path(__file__).resolve().parents[2] / "data" / "inactivity_reset.lock"
def _cn_today() -> date:
return cn_today()
def _touch_lock() -> None:
with contextlib.suppress(FileNotFoundError):
os.utime(_LOCK_PATH, None)
@contextlib.contextmanager
def _single_instance_lock(stale_after_sec: int) -> Iterator[bool]:
"""同机多进程保护:同一时间只允许一个清零 worker 运行。"""
_LOCK_PATH.parent.mkdir(parents=True, exist_ok=True)
fd: int | None = None
try:
try:
fd = os.open(str(_LOCK_PATH), os.O_CREAT | os.O_EXCL | os.O_WRONLY)
except FileExistsError:
try:
age = time.time() - _LOCK_PATH.stat().st_mtime
except FileNotFoundError:
age = stale_after_sec + 1
if age > stale_after_sec:
with contextlib.suppress(FileNotFoundError):
_LOCK_PATH.unlink()
try:
fd = os.open(str(_LOCK_PATH), os.O_CREAT | os.O_EXCL | os.O_WRONLY)
except FileExistsError:
fd = None
if fd is None:
yield False
return
os.write(fd, f"pid={os.getpid()} started_at={int(time.time())}\n".encode("ascii"))
yield True
finally:
if fd is not None:
os.close(fd)
with contextlib.suppress(FileNotFoundError):
_LOCK_PATH.unlink()
def _run_once_entry() -> dict:
"""跑一轮(预警 + 清零)。独立开 Session。"""
notifier = get_notifier(settings.INACTIVITY_NOTIFY_CHANNEL)
with SessionLocal() as db:
return inactivity_repo.run_once(
db,
notifier=notifier,
reset_days=settings.INACTIVITY_RESET_DAYS,
warn_stages=settings.inactivity_warn_stages,
today=_cn_today(),
dry_run=not settings.INACTIVITY_RESET_ENABLED, # ENABLED=false → 只记审计名单、不清
)
async def _run_loop() -> None:
interval = max(60, int(settings.INACTIVITY_RESET_CHECK_INTERVAL_SEC))
lock_stale_after = max(interval * 3, 1800)
with _single_instance_lock(lock_stale_after) as lock_acquired:
if not lock_acquired:
logger.warning("inactivity reset skipped: another worker owns lock")
return
await _run_locked_loop(interval)
async def _run_locked_loop(interval: int) -> None:
logger.info(
"inactivity reset worker started interval=%ss run_hour=%s mode=%s",
interval,
settings.INACTIVITY_RESET_RUN_HOUR,
"clear" if settings.INACTIVITY_RESET_ENABLED else "dry-run(audit-only)",
)
# 本进程上次跑过的北京日;None=尚未跑过本进程(当天到点即补)。
last_run: date | None = None
try:
while True:
try:
_touch_lock()
today = _cn_today()
hour = datetime.now(CN_TZ).hour
if last_run != today and hour >= int(settings.INACTIVITY_RESET_RUN_HOUR):
result = await asyncio.to_thread(_run_once_entry)
last_run = today
logger.info("inactivity reset done date=%s result=%s", today, result)
except SQLAlchemyError:
logger.exception("inactivity reset db error")
except Exception: # noqa: BLE001 - 后台任务不能因单次异常退出
logger.exception("inactivity reset unexpected error")
await asyncio.sleep(interval)
except asyncio.CancelledError:
logger.info("inactivity reset worker stopped")
raise
def start_inactivity_reset_worker() -> asyncio.Task | None:
# worker 常驻(不再有"完全关"档);INACTIVITY_RESET_ENABLED 只决定是否**真清**:
# false(默认)= 只记审计名单(dry-run,不动钱),true = 真清金币+现金。
return asyncio.create_task(_run_loop(), name="inactivity-reset")
async def stop_inactivity_reset_worker(task: asyncio.Task | None) -> None:
if task is None:
return
task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await task
+110
View File
@@ -0,0 +1,110 @@
"""接口指标埋点:有界事件队列 + 纯 ASGI 中间件。
每个 HTTP 请求测总耗时、抓路由模板 + 状态码,非阻塞塞进有界队列;由 observe_worker
后台批量上报到 OpenObserve。请求路径上无任何 I/O。未配置观测时中间件直接透传。
"""
from __future__ import annotations
import asyncio
import os
import time
from starlette.routing import Match
from app.core.config import settings
# 不采集的路径(纯噪音):健康检查。
_SKIP_PATHS = frozenset({"/health"})
# 未匹配路由(404/扫描器)归一到此,防维度爆炸。
_UNMATCHED = "__unmatched__"
# service 字段:与 logging.py 同源(LOG_SERVICE_NAME),默认 app-server。
_SERVICE = os.getenv("LOG_SERVICE_NAME", "app-server")
# 有界事件队列(懒创建,见 get_queue):首次取用时在运行中的 loop 里建,避免 import 期
# 无 loop 的边角问题;put_nowait/get_nowait 不需运行中的 loop → 可在无 loop 下测试。
_queue: asyncio.Queue[dict] | None = None
# 队列满时的丢弃计数,worker 定期取出打日志。
_dropped = 0
def get_queue() -> asyncio.Queue[dict]:
"""返回全局有界事件队列(懒创建)。测试可 monkeypatch 模块级 _queue 换成小队列。"""
global _queue
if _queue is None:
_queue = asyncio.Queue(maxsize=settings.OBSERVE_QUEUE_MAX)
return _queue
def take_dropped() -> int:
"""取出并清零累计丢弃数(供 worker 打点)。"""
global _dropped
n, _dropped = _dropped, 0
return n
def record_event(event: dict) -> None:
"""非阻塞入队;队列满则丢弃当前事件并计数。永不抛异常、永不阻塞请求。"""
global _dropped
try:
get_queue().put_nowait(event)
except asyncio.QueueFull:
_dropped += 1
def _resolve_route(scope) -> str:
"""从 scope 取路由模板(如 /things/{tid})。优先 scope['route'](现代 Starlette
路由后写入);取不到则手动匹配一次(老版本兜底);仍无 → __unmatched__(404/扫描器)。"""
route = scope.get("route")
path = getattr(route, "path", None)
if path:
return path
app_ = scope.get("app")
router = getattr(app_, "router", None)
for candidate in getattr(router, "routes", []):
try:
match, _ = candidate.matches(scope)
except Exception: # noqa: BLE001 - 匹配兜底,任一路由异常不影响整体
continue
if match == Match.FULL and getattr(candidate, "path", None):
return candidate.path
return _UNMATCHED
class RequestMetricsMiddleware:
"""纯 ASGI 中间件:测每个 http 请求耗时,记 method/route/status/duration。
放在最外层(main.py 里 CORS 之后 add),测到含 CORS 的完整耗时。未配置观测 → 透传。
"""
def __init__(self, app) -> None:
self.app = app
async def __call__(self, scope, receive, send) -> None:
if scope["type"] != "http" or not settings.observe_configured:
await self.app(scope, receive, send)
return
if scope.get("path") in _SKIP_PATHS:
await self.app(scope, receive, send)
return
start = time.perf_counter()
status_holder = {"status": 500} # 下游异常未产出 response 时兜底 500
async def send_wrapper(message) -> None:
if message["type"] == "http.response.start":
status_holder["status"] = message["status"]
await send(message)
try:
await self.app(scope, receive, send_wrapper)
finally:
duration_ms = (time.perf_counter() - start) * 1000.0
record_event({
"_timestamp": int(time.time() * 1_000_000), # µs,OpenObserve 时间列
"service": _SERVICE,
"env": settings.APP_ENV,
"method": scope.get("method", ""),
"route": _resolve_route(scope),
"status": status_holder["status"],
"duration_ms": round(duration_ms, 3),
})
+128
View File
@@ -0,0 +1,128 @@
"""接口指标后台上报 worker:批量 drain 事件队列 → POST 到 OpenObserve。
对齐 heartbeat_monitor_worker 等的 start_*/stop_* 形态。best-effort 遥测:catch 全部
异常,上报失败直接丢批不重试。未配置观测 → start 返回 None(不启动),整套 no-op。
"""
from __future__ import annotations
import asyncio
import contextlib
import logging
import httpx
from app.core.config import settings
from app.core.observe import get_queue, take_dropped
logger = logging.getLogger("shagua.observe")
# 上报用的 httpx client,start 时建、stop 时关。
_client: httpx.AsyncClient | None = None
async def _collect_batch() -> list[dict]:
"""等到 ≥1 条(或到 flush 间隔)后,连抽到 BATCH_MAX 条或抽空。超时且空 → 返回 []。"""
queue = get_queue()
batch: list[dict] = []
try:
first = await asyncio.wait_for(
queue.get(), timeout=settings.OBSERVE_FLUSH_INTERVAL_SEC
)
except asyncio.TimeoutError: # noqa: UP041 - 3.10 兼容:该版 wait_for 抛的 asyncio.TimeoutError ≠ 内置 TimeoutError
return batch
batch.append(first)
while len(batch) < settings.OBSERVE_BATCH_MAX:
try:
batch.append(queue.get_nowait())
except asyncio.QueueEmpty:
break
return batch
async def _post_batch(client: httpx.AsyncClient, batch: list[dict]) -> None:
"""POST 一批事件到 OpenObserve 的 _json ingest 端点。非 2xx 仅告警。"""
url = f"/api/{settings.OBSERVE_ORG}/{settings.OBSERVE_STREAM}/_json"
resp = await client.post(url, json=batch)
if resp.status_code >= 300:
logger.warning(
"observe ingest failed status=%s body=%s",
resp.status_code,
resp.text[:200],
)
async def _run_loop(client: httpx.AsyncClient) -> None:
try:
while True:
batch = await _collect_batch()
dropped = take_dropped()
if dropped:
logger.warning("observe dropped %d events (queue full)", dropped)
if not batch:
continue
try:
await _post_batch(client, batch)
except Exception: # noqa: BLE001 - best-effort 遥测,失败丢批不重试、不退出
logger.warning(
"observe post batch failed, dropped %d events",
len(batch),
exc_info=True,
)
except asyncio.CancelledError:
logger.info("observe worker stopped")
raise
def start_observe_worker() -> asyncio.Task | None:
"""启动上报 worker。未配置观测 → 返回 None(no-op)。约定每进程只调一次(lifespan)。"""
global _client
if not settings.observe_configured:
return None
if _client is not None:
# 约定 start 每进程只调一次;已启动则不重复建 client(避免泄漏旧连接池)。
logger.warning("observe worker already started; ignoring duplicate start")
return None
_client = httpx.AsyncClient(
base_url=settings.OBSERVE_ENDPOINT,
auth=(settings.OBSERVE_USER, settings.OBSERVE_PASSWORD),
timeout=settings.OBSERVE_TIMEOUT_SEC,
)
logger.info(
"observe worker started endpoint=%s org=%s stream=%s",
settings.OBSERVE_ENDPOINT,
settings.OBSERVE_ORG,
settings.OBSERVE_STREAM,
)
return asyncio.create_task(_run_loop(_client), name="observe-worker")
async def stop_observe_worker(task: asyncio.Task | None) -> None:
"""收尾:cancel worker → best-effort 发最后一批 → 关 client。"""
global _client
if task is None:
return
task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await task
dropped = take_dropped() # 收口:补记最后一个 flush 窗口累计的丢弃数,不让账丢在关停期
if dropped:
logger.warning("observe dropped %d events (queue full) before shutdown", dropped)
if _client is not None:
# worker 已停,安全 drain 剩余并 best-effort 发最后一批(短超时,不卡关停);
# 超过一批(BATCH_MAX)的剩余直接丢,不做多轮 flush(best-effort,关停从速)。
try:
queue = get_queue()
final: list[dict] = []
while len(final) < settings.OBSERVE_BATCH_MAX:
try:
final.append(queue.get_nowait())
except asyncio.QueueEmpty:
break
if final:
await asyncio.wait_for(
_post_batch(_client, final), timeout=settings.OBSERVE_TIMEOUT_SEC
)
except Exception: # noqa: BLE001 - 关停期尽力而为,失败忽略
pass
await _client.aclose()
_client = None
+93 -8
View File
@@ -9,29 +9,41 @@ from __future__ import annotations
import threading
import time
from typing import NamedTuple
from fastapi import HTTPException, Request, status
from app.core.config import settings
# key -> (window_start_ts, count)
_buckets: dict[str, tuple[float, int]] = {}
# key -> (window_start_ts, count, window_sec)
# 存每个 key 自己的 window_sec:_buckets 混着不同窗口(60s 广告 / 3600s 登录 / 86400s 日闸)的 key,
# GC 必须按各 key 自己的窗口判过期(见 [_purge_expired]),否则短窗口调用触发的 GC 会误删长窗口 key。
_buckets: dict[str, tuple[float, int, float]] = {}
_lock = threading.Lock()
_GC_THRESHOLD = 10000 # _buckets 超此阈值才顺手清过期 key(仿 sms.py;测试可 monkeypatch 调小强制每次扫)
def _purge_expired(now: float) -> None:
"""清过期 key(**仅在持有 _lock 时调用**)。按每个 key 自己存的 window_sec 判过期,而非调用方的窗口
—— _buckets 是全局共享、混着 60s(广告)/3600s(登录)/86400s(日闸)不同窗口的 key;若用调用方窗口,
高频的 60s 广告端点触发 GC 时会把本该活 3600s/86400s 的登录/日闸计数一并删掉,使其在规模上(超阈值才
触发本清理)被反复清零而失效。仅在超阈值时扫,低频、开销可忽略。"""
if len(_buckets) <= _GC_THRESHOLD:
return
for k in [k for k, (s, _, w) in _buckets.items() if now - s >= w]:
_buckets.pop(k, None)
def _hit(key: str, limit: int, window_sec: float) -> bool:
"""记一次访问。返回 True=放行,False=超限。"""
now = time.monotonic()
with _lock:
start, count = _buckets.get(key, (now, 0))
start, count, _ = _buckets.get(key, (now, 0, window_sec))
if now - start >= window_sec: # 窗口过期,重置
start, count = now, 0
count += 1
_buckets[key] = (start, count)
# 顺手清过期 key,防内存无限涨(低频访问足够)
if len(_buckets) > 10000:
for k in [k for k, (s, _) in _buckets.items() if now - s >= window_sec]:
_buckets.pop(k, None)
_buckets[key] = (start, count, window_sec)
_purge_expired(now) # 顺手清过期 key(按各自窗口),防内存无限涨
return count <= limit
@@ -83,3 +95,76 @@ def enforce_rate_limit(
status_code=status.HTTP_429_TOO_MANY_REQUESTS,
detail=detail,
)
# ===================== 先判 / 后记(只按「成功」计数)=====================
# _hit 是原子「判+记」:一调用就 +1,适合登录爆破(失败尝试也要计)。但对「短信发码」这类
# **只想给成功动作计数**的场景不合适 —— 被单号冷却挡下的重发没真发、没烧钱,不该占额度。
# 故拆成 _peek(只判不记)+ _commit(只记):check_rate_limits 先判 → 动作 → 成功后 record。
class RateLimitRule(NamedTuple):
"""一条限流规则。scope 区分不同闸(不同 key 前缀);同一 (subject, IP) 在 window_sec
内最多 limit 次,超限抛 429 用 detail 文案。
(scope, window_sec) 成对绑在一条规则里 —— check(先判)与 record(计数)复用同一条,
避免两处把窗口/scope 写歪导致 key 对不上。
"""
scope: str
limit: int
window_sec: float
detail: str = "操作过于频繁,请稍后再试"
def _peek(key: str, limit: int, window_sec: float) -> bool:
"""只读:当前窗口内是否还没到上限(count < limit)。**不改计数**。
与 [_commit] 配对实现「先判后记」——只在动作成功后才 _commit。"""
now = time.monotonic()
with _lock:
start, count, _ = _buckets.get(key, (now, 0, window_sec))
if now - start >= window_sec: # 窗口已过期 → 视作已重置(count 归零)
count = 0
return count < limit
def _commit(key: str, window_sec: float) -> None:
"""记一次访问(+1)。窗口过期则以本次为起点重置。仅在动作成功后调用。"""
now = time.monotonic()
with _lock:
start, count, _ = _buckets.get(key, (now, 0, window_sec))
if now - start >= window_sec: # 窗口过期,重置
start, count = now, 0
_buckets[key] = (start, count + 1, window_sec)
_purge_expired(now) # 顺手清过期 key(按各自窗口,同 [_hit])
def check_rate_limits(request: Request, subject: str, rules: list[RateLimitRule]) -> None:
"""【先判】一组限流:任一规则已达上限即抛 429,且**不改计数**。
配合 [record_rate_limits] 实现「只按成功计数」:先 check 所有闸(全未超才继续)→ 执行动作
→ 动作**成功后**再 record。动作被下游挡下(如短信单号冷却)、没真正发生时不 record → 不占额度。
key = `scope:subject:client_ip`(与 [enforce_rate_limit] 同款)。
"""
if not settings.RATE_LIMIT_ENABLED:
return
ip = _client_ip(request)
for rule in rules:
if not _peek(f"{rule.scope}:{subject}:{ip}", rule.limit, rule.window_sec):
raise HTTPException(
status_code=status.HTTP_429_TOO_MANY_REQUESTS,
detail=rule.detail,
)
def record_rate_limits(request: Request, subject: str, rules: list[RateLimitRule]) -> None:
"""【记一次】一组限流(每条规则 +1)。仅在动作成功后调用,与 [check_rate_limits] 配对。
⚠️ check→动作→record 非原子:并发突发下计数可能略超 limit(每个在途请求各 +1)。对
「防脚本/防轰炸」的安全网定位可接受;要精确配额需迁 Redis(见模块 docstring)。
"""
if not settings.RATE_LIMIT_ENABLED:
return
ip = _client_ip(request)
for rule in rules:
_commit(f"{rule.scope}:{subject}:{ip}", rule.window_sec)
+85
View File
@@ -87,6 +87,91 @@ def issue_token_pair(user_id: int) -> dict[str, Any]:
}
def create_bind_ticket(
*, openid: str, wechat_nickname: str | None, wechat_avatar_url: str | None
) -> str:
"""微信登录未命中 openid 时,签发短时"待绑手机"令牌,承载 openid + 微信昵称头像。
typ='wechat_bind'、sub=openid;有效期 settings.WECHAT_BIND_TICKET_EXPIRE_MINUTES 分钟。
与 access/refresh 用同一 JWT_SECRET_KEY 签名,靠 typ 区分,decode_bind_ticket 校验 typ。
"""
now = _now()
expire = now + timedelta(minutes=settings.WECHAT_BIND_TICKET_EXPIRE_MINUTES)
payload: dict[str, Any] = {
"sub": openid,
"typ": "wechat_bind",
"wnk": wechat_nickname,
"wav": wechat_avatar_url,
"iat": int(now.timestamp()),
"exp": int(expire.timestamp()),
}
return jwt.encode(payload, settings.JWT_SECRET_KEY, algorithm=settings.JWT_ALGORITHM)
def decode_bind_ticket(token: str) -> dict[str, Any]:
"""解析"待绑手机"令牌,校验签名/过期/类型。失败抛 TokenError。
返回 {'openid': str, 'wnk': str|None, 'wav': str|None}。
"""
try:
payload = jwt.decode(token, settings.JWT_SECRET_KEY, algorithms=[settings.JWT_ALGORITHM])
except jwt.ExpiredSignatureError as e:
raise TokenError("bind ticket expired") from e
except jwt.InvalidTokenError as e:
raise TokenError(f"invalid bind ticket: {e}") from e
if payload.get("typ") != "wechat_bind":
raise TokenError(f"wrong token type: want=wechat_bind got={payload.get('typ')}")
if "sub" not in payload:
raise TokenError("bind ticket missing sub")
return {"openid": payload["sub"], "wnk": payload.get("wnk"), "wav": payload.get("wav")}
def create_conflict_ticket(
*, openid: str, wechat_nickname: str | None, wechat_avatar_url: str | None, phone: str
) -> str:
"""手机号占用时签发的短时"冲突处理"令牌。
比 bind_ticket 多编码 **已验证的手机号 phone** —— 换绑/继续绑定只认它,证明"这对
openid/手机号刚在绑号时验证通过",免用户重输验证码,又堵住"拿自己 openid + 任意手机号
去夺号"的接管漏洞。typ='wechat_conflict';有效期复用 WECHAT_BIND_TICKET_EXPIRE_MINUTES。
"""
now = _now()
expire = now + timedelta(minutes=settings.WECHAT_BIND_TICKET_EXPIRE_MINUTES)
payload: dict[str, Any] = {
"sub": openid,
"typ": "wechat_conflict",
"wnk": wechat_nickname,
"wav": wechat_avatar_url,
"phn": phone,
"iat": int(now.timestamp()),
"exp": int(expire.timestamp()),
}
return jwt.encode(payload, settings.JWT_SECRET_KEY, algorithm=settings.JWT_ALGORITHM)
def decode_conflict_ticket(token: str) -> dict[str, Any]:
"""解析"冲突处理"令牌,校验签名/过期/类型。失败抛 TokenError。
返回 {'openid': str, 'wnk': str|None, 'wav': str|None, 'phone': str}。
"""
try:
payload = jwt.decode(token, settings.JWT_SECRET_KEY, algorithms=[settings.JWT_ALGORITHM])
except jwt.ExpiredSignatureError as e:
raise TokenError("conflict ticket expired") from e
except jwt.InvalidTokenError as e:
raise TokenError(f"invalid conflict ticket: {e}") from e
if payload.get("typ") != "wechat_conflict":
raise TokenError(f"wrong token type: want=wechat_conflict got={payload.get('typ')}")
if "sub" not in payload or "phn" not in payload:
raise TokenError("conflict ticket missing sub/phn")
return {
"openid": payload["sub"],
"wnk": payload.get("wnk"),
"wav": payload.get("wav"),
"phone": payload["phn"],
}
# ===================== 密码 hash(admin 后台账号用)=====================
# 用户侧是手机号+验证码登录,不存密码;仅 admin 账号用 username+password 登录。
-27
View File
@@ -1,27 +0,0 @@
"""微信截图比价 PoC:进程内存的"待比价"信号(单 worker 够用, 重启即失效, PoC 可接受)。
收图端点 set_pending(device_id, source) → 该设备下次心跳 pop_pending 取走并清除 →
心跳响应带回 → 前端弹选平台窗。只对 settings.WX_POC_TEST_DEVICE_ID 写入, 不碰其他设备。
后续接真识别 / openid↔device 绑定时整体替换本模块。
"""
from __future__ import annotations
import threading
_lock = threading.Lock()
_pending: dict[str, str] = {} # device_id -> source_platform(如 "meituan")
def set_pending(device_id: str, source_platform: str) -> None:
if not device_id:
return
with _lock:
_pending[device_id] = source_platform
def pop_pending(device_id: str) -> str | None:
"""取出并清除该设备的待比价信号;无则 None。心跳每帧调, 取到即消费(只弹一次)。"""
if not device_id:
return None
with _lock:
return _pending.pop(device_id, None)
+43
View File
@@ -0,0 +1,43 @@
"""不活跃预警通知器(可插拔)。
v1 仅日志占位(LogNotifier):现状无真实推送能力(极光只用于一键登录解密 + 设备心跳告警,
心跳 worker 也只打印),先把清零主流程 + 审计做扎实。后续实现同协议的 JPushNotifier /
SmsNotifier 即可替换,worker/repo 不改。
"""
from __future__ import annotations
import logging
from typing import Protocol
logger = logging.getLogger("shagua.inactivity")
class InactivityNotifier(Protocol):
channel: str
def warn(self, *, user_id: int, coin: int, cash_cents: int,
stage: int, days_until_reset: int) -> str:
"""发预警(只涉及会被清的金币 + 折算现金;邀请现金不清、不预警)。
返回状态:'sent' / 'failed' / 'placeholder'"""
...
class LogNotifier:
"""占位实现:只打印,不真推。参照 heartbeat_monitor_worker「本期先不接推送」先例。"""
channel = "log"
def warn(self, *, user_id: int, coin: int, cash_cents: int,
stage: int, days_until_reset: int) -> str:
logger.warning(
"[inactivity-warn] user=%s coin=%s cash_cents=%s stage=T-%s days_until_reset=%s",
user_id, coin, cash_cents, stage, days_until_reset,
)
return "placeholder"
def get_notifier(channel: str) -> InactivityNotifier:
"""按配置返回通知器。未实现的通道(jpush/sms)暂回退 LogNotifier 占位。"""
# 后续:if channel == "jpush": return JPushNotifier()
# if channel == "sms": return SmsNotifier()
return LogNotifier()
+2 -1
View File
@@ -13,7 +13,8 @@ worker / 多机时内存不共享 → 冷却、校验都会失效,届时迁移
防刷两层(短信花钱 + `/sms/send` 在登录前无法 JWT 鉴权):
1. 单号 `SMS_SEND_INTERVAL_SEC` 冷却(本文件)
2. 单设备(device_id)每小时频控(api 层 auth.sms_send 内 enforce_rate_limit)+ 极光控制台 IP 白名单/防轰炸(运维侧)。
2. 单设备(device_id)+ IP 每小时 / 每天频控(api 层 auth.sms_send 的 check/record_rate_limits,
**只按成功发码计数** —— 被本文件单号冷却挡下的重发不占额度)+ 极光控制台 IP 白名单/防轰炸(运维侧)。
⚠️ 原「单 IP 频控(rate_limit 依赖)」2026-06-26 按产品要求删除、改设备维度;但 device_id 客户端可伪造/轮换,
脚本轮换 id 能绕过本层 → 挡脚本狂发主要靠极光控制台侧(+ 可选 nginx 限流)。
⚠️ 原「单号每日上限」2026-07-03 按精简要求删除(mentor 定:登录风控只留单号冷却 + 单设备频控);
-100
View File
@@ -1,100 +0,0 @@
"""微信会话内容存档 SDK 封装(WeWorkFinanceSdk C 库的 ctypes 绑定 + 消息解密)。
企业微信「会话内容存档」是拿"成员↔外部用户私聊消息正文"(含美团小程序卡片)的官方途径:
主动轮询 GetChatData(seq) 拉取。每条消息两段密文:
- encrypt_random_key:用【企业自持 RSA 私钥】解出 AES 密钥
- encrypt_chat_msg :SDK DecryptData 用该密钥解出明文消息 JSON
本模块封装:加载 .so → Init → GetChatData → RSA 解密随机密钥 → DecryptData。纯外部依赖,
不含业务逻辑(识别 weapp/image → 打信号在 core/wx_finance_worker.py)。
.so 需放服务器(Linux `libWeWorkFinanceSdk_C.so`),路径 settings.WX_FINANCE_SDK_PATH。
⚠️ CDLL 在实例化时才加载(非 import 期),故本模块在无 .so 的机器上也能安全 import。
"""
from __future__ import annotations
import base64
import ctypes
import json
import logging
from typing import Any
from cryptography.hazmat.primitives.asymmetric import padding
from cryptography.hazmat.primitives.serialization import load_pem_private_key
logger = logging.getLogger("shagua.wx_finance")
class WxFinanceError(RuntimeError):
"""SDK 调用返回非 0 / 业务 errcode 时抛。"""
class WxFinanceSdk:
"""WeWorkFinanceSdk 薄封装。实例化即加载 .so + Init(需 corpid/secret 正确 + 服务器 IP 在可信IP)。"""
def __init__(self, sdk_path: str, corpid: str, secret: str, private_key_pem: bytes) -> None:
self._priv = load_pem_private_key(private_key_pem, password=None)
self._lib = ctypes.CDLL(sdk_path) # 缺 .so / 缺 libssl 依赖会在此抛 OSError
self._bind()
self._sdk = self._lib.NewSdk()
ret = self._lib.Init(self._sdk, corpid.encode(), secret.encode())
if ret != 0:
raise WxFinanceError(f"Init 失败 ret={ret}(检查 corpid / 会话存档 Secret / 可信IP)")
def _bind(self) -> None:
lib = self._lib
lib.NewSdk.restype = ctypes.c_void_p
lib.Init.argtypes = [ctypes.c_void_p, ctypes.c_char_p, ctypes.c_char_p]
lib.Init.restype = ctypes.c_int
lib.GetChatData.argtypes = [
ctypes.c_void_p, ctypes.c_ulonglong, ctypes.c_uint,
ctypes.c_char_p, ctypes.c_char_p, ctypes.c_int, ctypes.c_void_p,
]
lib.GetChatData.restype = ctypes.c_int
# ⚠️ DecryptData 不吃 sdk 句柄(与 GetChatData 不同, 它是纯解密函数)——只有 3 个参数,
# 多传 sdk 会让参数错位、encrypt_key 收到 sdk 指针 → DecryptData 返 10008(解析 encrypt_key 出错)。
lib.DecryptData.argtypes = [ctypes.c_char_p, ctypes.c_char_p, ctypes.c_void_p]
lib.DecryptData.restype = ctypes.c_int
lib.NewSlice.restype = ctypes.c_void_p
lib.FreeSlice.argtypes = [ctypes.c_void_p]
lib.GetContentFromSlice.argtypes = [ctypes.c_void_p]
lib.GetContentFromSlice.restype = ctypes.c_void_p
lib.GetSliceLen.argtypes = [ctypes.c_void_p]
lib.GetSliceLen.restype = ctypes.c_int
lib.DestroySdk.argtypes = [ctypes.c_void_p]
def _slice_bytes(self, slc: int) -> bytes:
ptr = self._lib.GetContentFromSlice(slc)
length = self._lib.GetSliceLen(slc)
return ctypes.string_at(ptr, length)
def get_chat_data(self, seq: int, limit: int = 1000, timeout: int = 10) -> list[dict[str, Any]]:
"""拉 seq 之后的消息(返回从 seq+1 起)。返回 chatdata 列表(每项含 seq/encrypt_random_key/encrypt_chat_msg)。"""
slc = self._lib.NewSlice()
try:
ret = self._lib.GetChatData(self._sdk, int(seq), int(limit), None, None, int(timeout), slc)
if ret != 0:
raise WxFinanceError(f"GetChatData 失败 ret={ret}")
data = json.loads(self._slice_bytes(slc).decode("utf-8"))
finally:
self._lib.FreeSlice(slc)
if data.get("errcode"):
raise WxFinanceError(f"GetChatData errcode={data.get('errcode')} {data.get('errmsg')}")
return data.get("chatdata", [])
def decrypt(self, encrypt_random_key_b64: str, encrypt_chat_msg: str) -> dict[str, Any]:
"""RSA 私钥解 encrypt_random_key → 得 AES 密钥 → DecryptData 解 encrypt_chat_msg → 明文 dict。"""
# 企业微信用你上传的 RSA 公钥(PKCS1)加密随机密钥, 这里用对应私钥解出, 原样交给 DecryptData。
aes_key = self._priv.decrypt(base64.b64decode(encrypt_random_key_b64), padding.PKCS1v15())
slc = self._lib.NewSlice()
try:
ret = self._lib.DecryptData(aes_key, encrypt_chat_msg.encode(), slc)
if ret != 0:
raise WxFinanceError(f"DecryptData 失败 ret={ret}")
return json.loads(self._slice_bytes(slc).decode("utf-8"))
finally:
self._lib.FreeSlice(slc)
def close(self) -> None:
if getattr(self, "_sdk", None):
self._lib.DestroySdk(self._sdk)
self._sdk = None
-126
View File
@@ -1,126 +0,0 @@
"""微信客服(企业微信)API 客户端:access_token 缓存 + sync_msg 拉消息 + 临时素材下载。
对应 api/v1/wx_kf.py:收到 `kf_msg_or_event` 回调后, 用这里的函数增量拉取用户消息
- access_token(7200s)进程内缓存, 提前 300s 刷新( worker 够用, 重启即失效)
- sync_msg 游标 next_cursor open_kfid 进程内存续(PoC 重启后从最近 3 天重拉, 可接受)
凭证来自 settings(WX_KF_CORP_ID / WX_KF_SECRET);纯外部 HTTP, 不含业务逻辑
"""
from __future__ import annotations
import asyncio
import logging
import time
from typing import Any
import httpx
from app.core.config import settings
logger = logging.getLogger("shagua.wx_kf")
_QYAPI = "https://qyapi.weixin.qq.com/cgi-bin"
# access_token 进程内缓存(单 worker)。token 空 / 未到刷新点直接用。
_token_lock = asyncio.Lock()
_token_cache: dict[str, Any] = {"token": "", "expire_at": 0.0}
# sync_msg 游标:open_kfid -> next_cursor(PoC 进程内存;重启从最近 3 天重拉)
_cursors: dict[str, str] = {}
async def _get_access_token() -> str:
"""取企业微信 access_token, 进程内缓存, 提前 300s 过期刷新。失败返回空串(调用方降级)。"""
now = time.time()
if _token_cache["token"] and _token_cache["expire_at"] - 300 > now:
return _token_cache["token"]
async with _token_lock:
now = time.time() # 拿锁后复检, 避免并发重复刷新
if _token_cache["token"] and _token_cache["expire_at"] - 300 > now:
return _token_cache["token"]
try:
async with httpx.AsyncClient(timeout=10) as client:
resp = await client.get(
f"{_QYAPI}/gettoken",
params={
"corpid": settings.WX_KF_CORP_ID,
"corpsecret": settings.WX_KF_SECRET,
},
)
data = resp.json()
except Exception:
logger.exception("wx_kf gettoken 请求失败")
return ""
if data.get("errcode"):
logger.warning(
"wx_kf gettoken errcode=%s errmsg=%s", data.get("errcode"), data.get("errmsg")
)
return ""
_token_cache["token"] = data.get("access_token", "")
_token_cache["expire_at"] = now + int(data.get("expires_in", 7200))
return _token_cache["token"]
async def sync_messages(token: str, open_kfid: str) -> list[dict[str, Any]]:
"""收到 kf_msg_or_event 回调后调用:用回调带的一次性 token 增量拉消息, has_more 循环拉完。
token: 回调事件里的 <Token>(消息拉取凭证, 非配置 Token;不传会有严格频控)
open_kfid:客服账号 id(回调里的 <OpenKfId>)
返回本次新拉到的 msg_list(已合并多页);游标按 open_kfid 进程内存续
"""
access_token = await _get_access_token()
if not access_token:
return []
out: list[dict[str, Any]] = []
cursor = _cursors.get(open_kfid, "")
try:
async with httpx.AsyncClient(timeout=15) as client:
for _ in range(20): # 最多 20 页护栏, 防异常时死循环
body: dict[str, Any] = {"cursor": cursor, "token": token, "limit": 1000}
if open_kfid:
body["open_kfid"] = open_kfid
resp = await client.post(
f"{_QYAPI}/kf/sync_msg",
params={"access_token": access_token},
json=body,
)
data = resp.json()
if data.get("errcode"):
logger.warning(
"wx_kf sync_msg errcode=%s errmsg=%s",
data.get("errcode"),
data.get("errmsg"),
)
break
out.extend(data.get("msg_list", []))
cursor = data.get("next_cursor", cursor)
_cursors[open_kfid] = cursor
if not data.get("has_more"):
break
except Exception:
logger.exception("wx_kf sync_msg 请求失败 open_kfid=%s", open_kfid)
return out
async def download_media(media_id: str) -> bytes | None:
"""临时素材下载(图片消息只给 media_id)。成功返回二进制, 失败返回 None。"""
if not media_id:
return None
access_token = await _get_access_token()
if not access_token:
return None
try:
async with httpx.AsyncClient(timeout=20) as client:
resp = await client.get(
f"{_QYAPI}/media/get",
params={"access_token": access_token, "media_id": media_id},
)
resp.raise_for_status()
# 成功=二进制文件流;失败=JSON(errcode)。据 Content-Type 区分。
ctype = resp.headers.get("Content-Type", "")
if "application/json" in ctype or "text/plain" in ctype:
logger.warning("wx_kf media/get 非文件响应: %s", resp.text[:200])
return None
return resp.content
except Exception:
logger.exception("wx_kf media/get 下载失败 media_id=%s", media_id)
return None
-69
View File
@@ -1,69 +0,0 @@
"""微信服务号消息接收回调的验签与解密(安全模式)。
服务号"服务器配置"选安全模式后:
- URL 接入验证(GET): sha1(sort(token, timestamp, nonce)) == signature 原样返回 echostr
- 消息(POST): body <Encrypt> 是密文;
msg_signature = sha1(sort(token, timestamp, nonce, encrypt))
密文 AES-256-CBC 解出: random(16B) + msg_len(4B big-endian) + msg + appid, PKCS7(=32) padding
只做验签 + 解密(收消息)被动回复(加密)MVP 不需要 收到后走客服消息异步回执
密钥/口令来自 settings(WX_MP_TOKEN / WX_MP_AES_KEY / WX_MP_APPID),本模块只做纯算法不读配置
"""
from __future__ import annotations
import base64
import hashlib
import hmac
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
def verify_url_signature(token: str, timestamp: str, nonce: str, signature: str) -> bool:
"""GET 接入验证:token/timestamp/nonce 三者字典序排序拼接后 sha1。"""
return _consteq(_sha1(token, timestamp, nonce), signature)
def verify_msg_signature(
token: str, timestamp: str, nonce: str, encrypt: str, msg_signature: str
) -> bool:
"""POST 消息验签:四者(含密文 encrypt)字典序排序拼接后 sha1。"""
return _consteq(_sha1(token, timestamp, nonce, encrypt), msg_signature)
def decrypt_message(aes_key_b64: str, expected_appid: str, encrypt_b64: str) -> str:
"""解密 <Encrypt> 密文, 返回明文消息 XML。appid 不符抛 ValueError。
aes_key_b64: EncodingAESKey(43 , 不含结尾 '='), '=' base64 解出 32 字节 AES-256 key
"""
aes_key = base64.b64decode(aes_key_b64 + "=") # 43 → 32 bytes
iv = aes_key[:16]
decryptor = Cipher(algorithms.AES(aes_key), modes.CBC(iv)).decryptor()
plain = decryptor.update(base64.b64decode(encrypt_b64)) + decryptor.finalize()
plain = _pkcs7_unpad(plain)
# random(16) + msg_len(4, big-endian) + msg(msg_len) + from_appid
content = plain[16:]
msg_len = int.from_bytes(content[:4], "big")
msg = content[4 : 4 + msg_len]
from_appid = content[4 + msg_len :].decode("utf-8")
if expected_appid and from_appid != expected_appid:
raise ValueError(f"appid mismatch: {from_appid!r} != {expected_appid!r}")
return msg.decode("utf-8")
def _sha1(*parts: str) -> str:
return hashlib.sha1("".join(sorted(parts)).encode("utf-8")).hexdigest()
def _consteq(a: str, b: str) -> bool:
return hmac.compare_digest(a, b)
def _pkcs7_unpad(data: bytes) -> bytes:
"""微信用块大小 32 的 PKCS7,末字节即 padding 长度(1..32)。越界则原样返回(容错)。"""
if not data:
return data
pad = data[-1]
if pad < 1 or pad > 32:
return data
return data[:-pad]
+16 -9
View File
@@ -19,7 +19,6 @@ from app.api.internal.app_version import router as internal_app_version_router
from app.api.internal.launch_confirm import router as internal_launch_confirm_router
from app.api.internal.price import router as internal_price_router
from app.api.internal.store import router as internal_store_router
from app.api.internal.wx_finance import router as internal_wx_finance_router
from app.api.v1.ad import router as ad_router
from app.api.v1.analytics import router as analytics_router
from app.api.v1.auth import router as auth_router
@@ -40,8 +39,6 @@ from app.api.v1.signin import router as signin_router
from app.api.v1.tasks import router as tasks_router
from app.api.v1.user import router as user_router
from app.api.v1.wallet import router as wallet_router
from app.api.v1.wx_kf import router as wx_kf_router
from app.api.v1.wx_mp import router as wx_mp_router
from app.api.v1.wxpay import router as wxpay_router
from app.core.config import settings
from app.core.daily_exchange_worker import (
@@ -52,7 +49,16 @@ from app.core.heartbeat_monitor_worker import (
start_heartbeat_monitor,
stop_heartbeat_monitor,
)
from app.core.inactivity_reset_worker import (
start_inactivity_reset_worker,
stop_inactivity_reset_worker,
)
from app.core.logging import setup_logging
from app.core.observe import RequestMetricsMiddleware
from app.core.observe_worker import (
start_observe_worker,
stop_observe_worker,
)
from app.core.pricebot_client import aclose_pricebot_client, get_pricebot_client
from app.core.withdraw_reconcile_worker import (
start_withdraw_reconcile_worker,
@@ -83,12 +89,16 @@ async def lifespan(_: FastAPI) -> AsyncIterator[None]:
reconcile_task = start_withdraw_reconcile_worker()
heartbeat_task = start_heartbeat_monitor()
daily_exchange_task = start_daily_exchange_worker()
observe_task = start_observe_worker()
inactivity_task = start_inactivity_reset_worker()
try:
yield
finally:
await stop_heartbeat_monitor(heartbeat_task)
await stop_withdraw_reconcile_worker(reconcile_task)
await stop_daily_exchange_worker(daily_exchange_task)
await stop_observe_worker(observe_task)
await stop_inactivity_reset_worker(inactivity_task)
await aclose_pricebot_client()
logger.info("shutting down")
@@ -110,6 +120,9 @@ if settings.cors_origins_list:
allow_headers=["*"],
)
# 接口指标埋点(放在 CORS 之后 = 最外层:测到含 CORS 的完整耗时)。未配置观测时中间件自 no-op。
app.add_middleware(RequestMetricsMiddleware)
@app.get("/health", tags=["meta"])
def health() -> dict[str, str]:
@@ -140,15 +153,9 @@ app.include_router(internal_price_router)
app.include_router(internal_store_router)
app.include_router(internal_app_version_router)
app.include_router(internal_launch_confirm_router)
# 会话存档独立 poller → 打比价信号(server→server, X-Internal-Secret;轮询进程在 app 外, 见 scripts/wx_finance_poller.py)
app.include_router(internal_wx_finance_router)
app.include_router(platform_router)
# CPS 群发短链跳转 /c/{code}(公网无鉴权:记点击 → 302 跳美团)
app.include_router(cps_redirect_router)
# 微信服务号消息接收回调 /wx/mp/callback(公网无鉴权:微信服务器验签, 截图比价入口)
app.include_router(wx_mp_router)
# 微信客服(企业微信)消息接收回调 /wx/kf/callback(服务号版的平行实现, 收事件→sync_msg 拉图)
app.include_router(wx_kf_router)
# 用户上传文件(头像)静态服务。生产可改由 nginx 直接 serve MEDIA_ROOT。
_media_root = Path(settings.MEDIA_ROOT)
+5
View File
@@ -27,11 +27,16 @@ from app.models.coupon_state import ( # noqa: F401
CouponSession,
)
from app.models.feedback import Feedback # noqa: F401
from app.models.inactivity import ( # noqa: F401
InactivityNotificationLog,
InactivityResetLog,
)
from app.models.invite import InviteRelation # noqa: F401
from app.models.invite_fingerprint import InviteFingerprint # noqa: F401
from app.models.launch_confirm_sample import LaunchConfirmSample # noqa: F401
from app.models.meituan_coupon import MeituanCoupon # noqa: F401
from app.models.onboarding import OnboardingCompletion # noqa: F401
from app.models.phone_rebind_log import PhoneRebindLog # noqa: F401
from app.models.ops_marquee_seed import OpsMarqueeSeed # noqa: F401
from app.models.ops_stat_config import OpsStatConfig # noqa: F401
from app.models.price_observation import PriceObservation # noqa: F401
+7 -1
View File
@@ -15,7 +15,7 @@ from __future__ import annotations
from datetime import datetime
from sqlalchemy import JSON, BigInteger, DateTime, Integer, String, func
from sqlalchemy import JSON, BigInteger, DateTime, Index, Integer, String, func
from sqlalchemy.orm import Mapped, mapped_column
from app.db.base import Base
@@ -23,6 +23,12 @@ from app.db.base import Base
class AnalyticsEvent(Base):
__tablename__ = "analytics_event"
__table_args__ = (
# 活跃口径聚合热点(activity.active_event_condition + last_active_subqueries):
# 按 (event,page) 过滤 首页可见(show/home)∪比价∪领券,再 group by user_id 取
# max(created_at)。覆盖索引 → 该聚合走 index-only,避免高频 show 事件全表扫。
Index("ix_analytics_event_active", "event", "page", "user_id", "created_at"),
)
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
+58
View File
@@ -0,0 +1,58 @@
"""15 天不活跃清零相关表。
- inactivity_reset_log:每次清零一行,记清零前三桶余额快照 + 原因 + 判定时活跃时间/不活跃天数,
供纠纷排查(需求①)清零同时另写 2 条钱包流水(金币 + 折算现金,biz_type=inactivity_reset),
资金流可逐笔回溯**邀请现金是产品红线不清零**,invite_cash_balance_cents_before 仅为清零时
仍保留的邀请现金快照(便于排查非被清金额; wallet.CoinAccount 注释)
- inactivity_notification_log:每次预警一行,记推送时余额快照 + 档位 + 通道 + 状态,
兼作"预警去重"依据(created_at > last_active)"待推送"占位 outbox(v1 通道=log)
append-only,不更新user_id 只索引不设外键( analytics_event,避免删用户级联/历史留痕)
"""
from __future__ import annotations
from datetime import datetime
from sqlalchemy import DateTime, Integer, String, func
from sqlalchemy.orm import Mapped, mapped_column
from app.db.base import Base
class InactivityResetLog(Base):
__tablename__ = "inactivity_reset_log"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
user_id: Mapped[int] = mapped_column(Integer, index=True, nullable=False)
coin_balance_before: Mapped[int] = mapped_column(Integer, nullable=False)
cash_balance_cents_before: Mapped[int] = mapped_column(Integer, nullable=False)
invite_cash_balance_cents_before: Mapped[int] = mapped_column(Integer, nullable=False)
last_active_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
inactive_days: Mapped[int] = mapped_column(Integer, nullable=False)
reason: Mapped[str] = mapped_column(String(32), nullable=False)
reset_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), index=True, nullable=False
)
def __repr__(self) -> str: # pragma: no cover
return f"<InactivityResetLog id={self.id} user_id={self.user_id} coin={self.coin_balance_before}>"
class InactivityNotificationLog(Base):
__tablename__ = "inactivity_notification_log"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
user_id: Mapped[int] = mapped_column(Integer, index=True, nullable=False)
stage: Mapped[int] = mapped_column(Integer, nullable=False) # 提前天数档(如 7 / 2)
inactive_days: Mapped[int] = mapped_column(Integer, nullable=False)
coin_balance: Mapped[int] = mapped_column(Integer, nullable=False)
cash_balance_cents: Mapped[int] = mapped_column(Integer, nullable=False)
invite_cash_balance_cents: Mapped[int] = mapped_column(Integer, nullable=False)
channel: Mapped[str] = mapped_column(String(16), nullable=False) # log / jpush / sms
status: Mapped[str] = mapped_column(String(16), nullable=False) # placeholder / sent / failed
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), index=True, nullable=False
)
def __repr__(self) -> str: # pragma: no cover
return f"<InactivityNotificationLog id={self.id} user_id={self.user_id} stage={self.stage}>"
+31
View File
@@ -0,0 +1,31 @@
"""手机号换绑台账。
记录"手机号从老账号被夺走、重建为新账号(X 注销 → Y)"这一破坏性事件,支撑"一个手机号
30 天内最多换绑一次"的限制。手机号级、渠道无关(source 标来源);普通微信绑定不写此表。
M2 spec §4.1
"""
from __future__ import annotations
from datetime import datetime
from sqlalchemy import DateTime, Integer, String, func
from sqlalchemy.orm import Mapped, mapped_column
from app.db.base import Base
class PhoneRebindLog(Base):
__tablename__ = "phone_rebind_log"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
# 被换绑的真实手机号(注意:存真实号,不是老账号被腾号后的 deleted_<id>)
phone: Mapped[str] = mapped_column(String(20), index=True, nullable=False)
# 被注销的老账号 X;P 换绑时已被腾空(极边界)则为空
old_user_id: Mapped[int | None] = mapped_column(Integer, nullable=True)
# 换绑后新建的账号 Y
new_user_id: Mapped[int] = mapped_column(Integer, nullable=False)
# 换绑来源。手机号级配额、渠道无关,留字段给未来其他换绑路径共用同一份 30 天限制。
source: Mapped[str] = mapped_column(String(32), nullable=False, default="wechat_conflict")
rebound_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), index=True, nullable=False
)
+101
View File
@@ -0,0 +1,101 @@
"""活跃口径唯一真源:worker(不活跃清零)与 admin(最近活跃/DAU)共用,防两处漂移。
口径 = max(User.created_at, AnalyticsEvent[首页可见 show/home + 比价 + 领券], CouponPromptEngagement[claim_started])
**不含 last_login_at**(登录/re-login 不代表在用 App);created_at 为恒非空基线
清零/预警按北京自然日 0 点对齐( reset_cutoff)
"""
from __future__ import annotations
from datetime import date, datetime, timedelta, timezone
from sqlalchemy import and_, func, or_, select
from sqlalchemy.orm import Session
from app.core.rewards import CN_TZ, cn_today
from app.models.analytics_event import AnalyticsEvent
from app.models.coupon_state import CouponPromptEngagement
# —— 活跃口径事件(与"用户管理"口径一致)——
# 首页可见:前端埋点 event=show + page=home(组合判定,单个 event 名不足以区分,见
# active_event_condition);其余为纯 event 名。
HOME_VIEW_EVENT = "show"
HOME_VIEW_PAGE = "home"
COMPARE_START_EVENT = "real_compare_start" # 发起比价(含浮窗触发)
COUPON_START_EVENT = "real_coupon_start" # 发起领券
# 纯 event 名即可判定的活跃事件(首页可见是 event+page 组合、不在此列)
ACTIVE_EVENTS = (COMPARE_START_EVENT, COUPON_START_EVENT)
ACTIVE_ENGAGE_TYPE = "claim_started" # coupon_prompt_engagement 一键领取
def active_event_condition():
"""analytics_event 中算"活跃"的行为过滤:首页可见(event=show & page=home)
发起比价 发起领券worker 子查询与 admin 展示共用,单一真源"""
return or_(
and_(AnalyticsEvent.event == HOME_VIEW_EVENT, AnalyticsEvent.page == HOME_VIEW_PAGE),
AnalyticsEvent.event.in_(ACTIVE_EVENTS),
)
def as_utc(value: datetime) -> datetime:
"""任意 datetime → tz-aware UTC(无时区按 UTC 解释)。用于与 DateTime(timezone=True) 列比较,
比较绝对时刻与会话时区无关(口径同 admin queries._as_utc)"""
if value.tzinfo is None:
return value.replace(tzinfo=timezone.utc)
return value.astimezone(timezone.utc)
def norm_utc(dt: datetime | None) -> datetime | None:
"""naive 视为 UTC 补 tzinfo(SQLite 读回 naive、PG 读回 aware,混着 max() 会 TypeError)。"""
if dt is None:
return None
return dt if dt.tzinfo is not None else dt.replace(tzinfo=timezone.utc)
def cn_midnight_utc(d: date) -> datetime:
"""北京 d 日 00:00 → tz-aware UTC datetime。"""
return as_utc(datetime(d.year, d.month, d.day, tzinfo=CN_TZ))
def reset_cutoff(reset_days: int, today: date | None = None) -> datetime:
"""应清零边界(tz-aware UTC):last_active < 此值 ⟺ 距末次活跃已满 reset_days 天(北京 0 点对齐)。
= 北京 00:00 of (today (reset_days 1)):reset_days=15today=1/20 北京 1/6 00:00"""
today = today or cn_today()
return cn_midnight_utc(today - timedelta(days=reset_days - 1))
def last_active_subqueries(db: Session):
"""两个按 user_id 预聚合的派生表:最近活跃事件(见 active_event_condition)、
最近领券发起(claim_started)返回 (ev_sub, eng_sub)口径同 admin,LEFT JOIN """
ev_sub = (
select(
AnalyticsEvent.user_id.label("user_id"),
func.max(AnalyticsEvent.created_at).label("last_at"),
)
.where(AnalyticsEvent.user_id.is_not(None), active_event_condition())
.group_by(AnalyticsEvent.user_id)
.subquery()
)
eng_sub = (
select(
CouponPromptEngagement.user_id.label("user_id"),
func.max(CouponPromptEngagement.created_at).label("last_at"),
)
.where(
CouponPromptEngagement.user_id.is_not(None),
CouponPromptEngagement.engage_type == ACTIVE_ENGAGE_TYPE,
)
.group_by(CouponPromptEngagement.user_id)
.subquery()
)
return ev_sub, eng_sub
def last_active_expr(base_col, ev_sub, eng_sub, dialect: str):
"""max(base_col, 最近活跃事件, 最近领券) 的 SQL 表达式。PG 用 greatest、SQLite 用 max。
子聚合缺失(未命中) coalesce base_col(= User.created_at,恒非空基线)"""
greatest = func.greatest if dialect == "postgresql" else func.max
return greatest(
base_col,
func.coalesce(ev_sub.c.last_at, base_col),
func.coalesce(eng_sub.c.last_at, base_col),
)
+198
View File
@@ -0,0 +1,198 @@
"""15 天不活跃清零业务逻辑(纯同步,可单测)。worker 只是它的 asyncio 外壳。
活跃口径复用 app.repositories.activity;清零走 wallet.grant_*(负数出账写流水 commit)
逐用户独立事务,一个失败不影响其余
"""
from __future__ import annotations
import logging
from datetime import date, datetime
from sqlalchemy import or_, select
from sqlalchemy.exc import SQLAlchemyError
from sqlalchemy.orm import Session
from app.core.rewards import CN_TZ
from app.integrations.notifier import InactivityNotifier
from app.models.inactivity import InactivityNotificationLog, InactivityResetLog
from app.models.user import User
from app.models.wallet import CoinAccount
from app.repositories import activity
from app.repositories import wallet as wallet_repo
logger = logging.getLogger("shagua.inactivity")
RESET_BIZ_TYPE = "inactivity_reset"
RESET_REMARK = "15天不活跃清零"
# 清零候选口径:金币或折算现金有余额即入选。**邀请现金不算**——它是产品红线、不清零
# (见 wallet.CoinAccount 注释),只有邀请现金余额的用户没有可清项,故不入选。
_ANY_BALANCE = or_(
CoinAccount.coin_balance > 0,
CoinAccount.cash_balance_cents > 0,
)
def _base_query(db: Session):
"""select(user_id, last_active, 三桶余额),join CoinAccount + 两活跃子查询。"""
ev_sub, eng_sub = activity.last_active_subqueries(db)
dialect = db.get_bind().dialect.name
last_active = activity.last_active_expr(User.created_at, ev_sub, eng_sub, dialect)
stmt = (
select(
User.id.label("user_id"),
last_active.label("last_active"),
CoinAccount.coin_balance,
CoinAccount.cash_balance_cents,
CoinAccount.invite_cash_balance_cents,
)
.join(CoinAccount, CoinAccount.user_id == User.id)
.outerjoin(ev_sub, ev_sub.c.user_id == User.id)
.outerjoin(eng_sub, eng_sub.c.user_id == User.id)
)
return stmt, last_active
def _cn_date(dt: datetime) -> date:
"""datetime → 北京自然日(naive 视为 UTC)。"""
return activity.norm_utc(dt).astimezone(CN_TZ).date()
def _inactive_days(last_active: datetime, today: date) -> int:
return (today - _cn_date(last_active)).days
def select_inactive_users(db: Session, *, cutoff: datetime):
"""应清零用户:last_active < cutoff 且金币/折算现金有余额(邀请现金不清、不计)。
返回 Row 列表(值已快照,可跨 commit)"""
stmt, last_active = _base_query(db)
stmt = stmt.where(_ANY_BALANCE, last_active < activity.as_utc(cutoff))
return db.execute(stmt).all()
def clear_user(db: Session, *, user_id: int, last_active: datetime, inactive_days: int,
reason: str, dry_run: bool = False) -> bool:
"""单用户清零(独立事务、行锁)。金币 + 折算现金归零 + 写审计 + 2 条流水;**邀请现金不清**
(产品红线, wallet.CoinAccount 注释),仅作快照记入审计返回是否真处理了(有可清余额)
dry_run=True:**只写审计名单不动钱不写流水**(灰度看名单) streak 去重 streak
已记过(reset_at > last_active)就跳,避免 worker 每日重复记"""
acc = wallet_repo.get_or_create_account(db, user_id, commit=False, lock=True)
coin, cash, invite = acc.coin_balance, acc.cash_balance_cents, acc.invite_cash_balance_cents
if coin == 0 and cash == 0: # 邀请现金不清,故不算"有可清余额"
return False
if dry_run and db.execute(
select(InactivityResetLog.id).where(
InactivityResetLog.user_id == user_id,
InactivityResetLog.reset_at > activity.as_utc(last_active),
).limit(1)
).first():
return False # dry-run:本 streak 已记过审计,不重复记
log = InactivityResetLog(
user_id=user_id, coin_balance_before=coin, cash_balance_cents_before=cash,
invite_cash_balance_cents_before=invite, last_active_at=activity.norm_utc(last_active),
inactive_days=inactive_days, reason=reason,
)
db.add(log)
db.flush() # 拿 log.id 作 ref_id 交叉链接审计↔流水
if not dry_run: # dry-run 只记审计名单,不真出账
ref = str(log.id)
if coin:
wallet_repo.grant_coins(db, user_id, -coin, biz_type=RESET_BIZ_TYPE, ref_id=ref, remark=RESET_REMARK)
if cash:
wallet_repo.grant_cash(db, user_id, -cash, biz_type=RESET_BIZ_TYPE, ref_id=ref, remark=RESET_REMARK)
# 邀请现金(invite_cash_balance_cents)刻意不动:两本账物理隔离、邀请金是产品红线。
db.commit()
return True
def run_reset_once(db: Session, *, reset_days: int, today: date, dry_run: bool = False) -> dict:
"""扫一轮清零。逐用户独立 commit,失败隔离。dry_run=True 只记审计名单、不动钱(见 clear_user)。"""
stats = {"scanned": 0, "cleared": 0, "failed": 0}
cutoff = activity.reset_cutoff(reset_days, today)
reason = f"inactive_{reset_days}d" + ("_dryrun" if dry_run else "")
rows = select_inactive_users(db, cutoff=cutoff) # 先物化,避免边遍历边 commit
for row in rows:
stats["scanned"] += 1
idays = _inactive_days(row.last_active, today)
try:
if clear_user(db, user_id=row.user_id, last_active=row.last_active,
inactive_days=idays, reason=reason, dry_run=dry_run):
stats["cleared"] += 1
except SQLAlchemyError:
db.rollback()
stats["failed"] += 1
return stats
def select_warn_candidates(db: Session, *, clear_cutoff: datetime, warn_hi: datetime):
"""预警候选:clear_cutoff <= last_active < warn_hi 且有可清余额(即已进预警窗、尚未到清零)。"""
stmt, last_active = _base_query(db)
stmt = stmt.where(
_ANY_BALANCE,
last_active >= activity.as_utc(clear_cutoff),
last_active < activity.as_utc(warn_hi),
)
return db.execute(stmt).all()
def run_warn_once(db: Session, notifier: InactivityNotifier, *,
reset_days: int, warn_stages: list[int], today: date) -> dict:
"""扫一轮预警。每人取"最紧急的已到达档",按 streak 去重(notification_log.created_at > last_active)。
预警只涉及会被清的金币 + 折算现金;邀请现金不清不预警(仅在 notification_log 记快照)
逐用户 try/except 隔离:单用户通知器抛错 / DB 错不阻断其余,也绝不能拖累后续清零"""
stats = {"warned": 0, "warn_skipped": 0, "warn_failed": 0}
if not warn_stages:
return stats
clear_cutoff = activity.reset_cutoff(reset_days, today) # 到此即清零,不再预警
warn_hi = activity.reset_cutoff(reset_days - max(warn_stages), today) # 最早预警档边界
ascending = sorted(warn_stages) # 最紧急(最小 k)在前
for row in select_warn_candidates(db, clear_cutoff=clear_cutoff, warn_hi=warn_hi):
idays = _inactive_days(row.last_active, today)
stage = next((k for k in ascending if idays >= reset_days - k), None)
if stage is None: # 防御:候选已在预警窗内、stage 必命中,此分支实际不可达
continue
try:
already = db.execute(
select(InactivityNotificationLog.id).where(
InactivityNotificationLog.user_id == row.user_id,
InactivityNotificationLog.stage == stage,
InactivityNotificationLog.created_at > activity.as_utc(row.last_active),
).limit(1)
).first()
if already:
stats["warn_skipped"] += 1
continue
status = notifier.warn(
user_id=row.user_id, coin=row.coin_balance, cash_cents=row.cash_balance_cents,
stage=stage, days_until_reset=reset_days - idays,
)
db.add(InactivityNotificationLog(
user_id=row.user_id, stage=stage, inactive_days=idays,
coin_balance=row.coin_balance, cash_balance_cents=row.cash_balance_cents,
invite_cash_balance_cents=row.invite_cash_balance_cents, # 快照,不参与"将清"额度
channel=notifier.channel, status=status,
))
db.commit()
stats["warned"] += 1
except Exception: # noqa: BLE001 - 单用户预警失败(通知器抛错/DB 错)隔离,不阻断其余、不拖累清零
db.rollback()
stats["warn_failed"] += 1
return stats
def run_once(db: Session, *, notifier: InactivityNotifier, reset_days: int,
warn_stages: list[int], today: date, dry_run: bool = False) -> dict:
"""一轮完整任务:先预警(阶段 A)再清零(阶段 B)。返回合并统计。
预警整段异常也**绝不阻塞清零**清零是核心不可逆资金操作,不能被通知故障拖住
dry_run=True(灰度默认):只记审计名单不清**也不预警**(不通知一个不会发生的清零)"""
warn = {"warned": 0, "warn_skipped": 0, "warn_failed": 0}
if not dry_run:
try:
warn = run_warn_once(db, notifier, reset_days=reset_days, warn_stages=warn_stages, today=today)
except Exception: # noqa: BLE001 - 预警阶段整体失败(如候选查询失败)也要继续清零
logger.exception("inactivity warn phase failed; proceeding to reset")
db.rollback()
warn = {"warned": 0, "warn_skipped": 0, "warn_failed": 0, "warn_phase_error": 1}
reset = run_reset_once(db, reset_days=reset_days, today=today, dry_run=dry_run)
return {**warn, **reset}
+39
View File
@@ -0,0 +1,39 @@
"""手机号换绑台账(phone_rebind_log)的查询与写入。见 M2 spec §4.1。"""
from __future__ import annotations
import math
from datetime import datetime, timedelta, timezone
from sqlalchemy import func, select
from sqlalchemy.orm import Session
from app.models.phone_rebind_log import PhoneRebindLog
def rebound_within_days(db: Session, phone: str, days: int) -> bool:
"""该手机号在最近 days 天内是否换绑过(命中 → 禁止再次换绑)。"""
since = datetime.now(timezone.utc) - timedelta(days=days)
stmt = (
select(PhoneRebindLog.id)
.where(PhoneRebindLog.phone == phone, PhoneRebindLog.rebound_at >= since)
.limit(1)
)
return db.execute(stmt).first() is not None
def remaining_block_days(db: Session, phone: str, days: int) -> int:
"""距离该手机号可再次换绑还剩几天(向上取整;无记录返回 0)。"""
last = db.execute(
select(func.max(PhoneRebindLog.rebound_at)).where(PhoneRebindLog.phone == phone)
).scalar_one_or_none()
if last is None:
return 0
if last.tzinfo is None: # SQLite 取回 naive datetime,按 UTC 归一
last = last.replace(tzinfo=timezone.utc)
remaining = (last + timedelta(days=days) - datetime.now(timezone.utc)).total_seconds()
return max(0, math.ceil(remaining / 86400))
def add_rebind_log(db: Session, *, phone: str, old_user_id: int | None, new_user_id: int, source: str) -> None:
"""写一条换绑台账(**不 commit**,交给调用方 rebind_account 的单事务)。"""
db.add(PhoneRebindLog(phone=phone, old_user_id=old_user_id, new_user_id=new_user_id, source=source))
+132
View File
@@ -12,6 +12,7 @@ from sqlalchemy import select
from sqlalchemy.orm import Session
from app.models.user import User
from app.repositories import phone_rebind
# ===== 创建时分配的标识:用户名(对外展示账号 ID)+ 默认昵称 =====
@@ -58,6 +59,17 @@ def is_default_nickname(nickname: str | None) -> bool:
)
def apply_wechat_display_identity(
user: User, *, wechat_nickname: str | None, wechat_avatar_url: str | None
) -> None:
"""§10:用已有账号绑微信时,仅当展示字段仍为默认才用微信昵称/头像替换(两规则独立);
自定义(改过昵称/传过头像)则保留只改内存对象,由调用方 commit"""
if is_default_nickname(user.nickname) and wechat_nickname:
user.nickname = wechat_nickname
if user.avatar_url is None and wechat_avatar_url:
user.avatar_url = wechat_avatar_url
def get_user_by_username(db: Session, username: str) -> User | None:
return db.execute(
select(User).where(User.username == username)
@@ -86,6 +98,85 @@ def get_user_by_phone(db: Session, phone: str) -> User | None:
return db.execute(stmt).scalar_one_or_none()
def get_user_by_wechat_openid(db: Session, openid: str) -> User | None:
stmt = select(User).where(User.wechat_openid == openid)
return db.execute(stmt).scalar_one_or_none()
def touch_last_login(db: Session, user: User) -> User:
"""openid 命中登录时更新 last_login_at(手机号登录在 upsert_user_for_login 里已更新)。"""
user.last_login_at = datetime.now(timezone.utc)
db.commit()
db.refresh(user)
return user
def attach_wechat_to_user(
db: Session, user: User, *, openid: str, wechat_nickname: str | None, wechat_avatar_url: str | None
) -> User:
"""继续绑定:把微信 openid + 微信源字段并入已存在账号(调用方保证 user.wechat_openid 为空)。
wechat_openid / wechat_nickname / wechat_avatar_url,并按 §10 规则回填展示字段:
仅当昵称仍为默认值(is_default_nickname)时用微信昵称替换,仅当头像为 null 时用微信头像替换;
用户已自定义的展示昵称/头像始终保留,两规则相互独立
openid 唯一约束(O 期间被别处绑走,极罕见)时由调用方捕获 IntegrityError 兜底降级为"只登入不绑"
"""
user.wechat_openid = openid
user.wechat_nickname = wechat_nickname
user.wechat_avatar_url = wechat_avatar_url
user.last_login_at = datetime.now(timezone.utc)
apply_wechat_display_identity(user, wechat_nickname=wechat_nickname, wechat_avatar_url=wechat_avatar_url)
db.commit()
db.refresh(user)
return user
def _build_wechat_user(
db: Session,
*,
phone: str,
openid: str,
wechat_nickname: str | None,
wechat_avatar_url: str | None,
) -> User:
"""构造并 db.add 一个微信账号行(register_channel='wechat',展示昵称头像取微信,缺则默认),
** commit**create_wechat_user rebind_account 共用,保证建号逻辑单一来源"""
user = User(
phone=phone,
username=_gen_unique_username(db),
nickname=wechat_nickname or _gen_nickname(),
avatar_url=wechat_avatar_url,
register_channel="wechat",
wechat_openid=openid,
wechat_nickname=wechat_nickname,
wechat_avatar_url=wechat_avatar_url,
last_login_at=datetime.now(timezone.utc),
)
db.add(user)
return user
def create_wechat_user(
db: Session,
*,
phone: str,
openid: str,
wechat_nickname: str | None,
wechat_avatar_url: str | None,
) -> User:
"""微信登录新建账号(未占用分支)。见 _build_wechat_user。
openid 唯一约束是并发/重复绑定的最终防线(极罕见,openid wechat-login 刚查过为空)
"""
user = _build_wechat_user(
db, phone=phone, openid=openid,
wechat_nickname=wechat_nickname, wechat_avatar_url=wechat_avatar_url,
)
db.commit()
db.refresh(user)
return user
def upsert_user_for_login(
db: Session,
*,
@@ -154,3 +245,44 @@ def soft_delete_account(db: Session, user: User) -> None:
# 释放邀请码唯一槽
user.invite_code = None
db.commit()
def rebind_account(
db: Session,
*,
phone: str,
openid: str,
wechat_nickname: str | None,
wechat_avatar_url: str | None,
source: str = "wechat_conflict",
) -> User:
"""换绑:**单事务内**注销老账号 X(腾出手机号)+ 用该号建全新微信账号 Y + 写换绑台账。
- 老账号可能已不存在(P 被腾空) old_user_id=None,直接建 Y(幂等更稳)
- 手机号唯一约束靠时序:先把 X.phone 改名并 flush 腾号,再插 Y
- 全程不中途 commit,任一步失败整体回滚,绝不出现"X 删了 Y 没建"
X 的字段变更等价 soft_delete_account(软删 + 匿名化 + 释放 openid/邀请码唯一槽),但不在此 commit
"""
old = get_user_by_phone(db, phone)
old_id = old.id if old is not None else None
if old is not None:
old.status = "deleted"
old.phone = f"deleted_{old.id}"
old.nickname = None
old.avatar_url = None
old.wechat_openid = None
old.wechat_nickname = None
old.wechat_avatar_url = None
old.invite_code = None
db.flush() # 先落 phone 改名,腾出手机号唯一约束,才能给 Y 用
new_user = _build_wechat_user(
db, phone=phone, openid=openid,
wechat_nickname=wechat_nickname, wechat_avatar_url=wechat_avatar_url,
)
db.flush() # 拿 new_user.id
phone_rebind.add_rebind_log(
db, phone=phone, old_user_id=old_id, new_user_id=new_user.id, source=source
)
db.commit()
db.refresh(new_user)
return new_user
+12 -4
View File
@@ -20,6 +20,7 @@ from app.core.config import settings
from app.core.rewards import COIN_PER_CENT, coins_to_cents
from app.integrations import wxpay
from app.models.user import User
from app.repositories.user import apply_wechat_display_identity
from app.models.wallet import (
CashTransaction,
CoinAccount,
@@ -34,6 +35,10 @@ _WX_STATE_SUCCESS = "SUCCESS"
_WX_STATE_FAILED = {"FAIL", "CANCELLED", "CLOSED"}
_WX_STATE_WAIT_CONFIRM = "WAIT_USER_CONFIRM" # 用户还没在微信确认页确认
_WITHDRAW_ACTIVE_STATUSES = {"reviewing", "pending"}
# 占用新人档「一次性」资格的提现状态:进行中(reviewing/pending)或成功打款(success)。
# 被拒/转账失败/解绑退回(rejected/failed,均已退款、钱没到手)不在此列 → 新人档恢复可提
# (2026-07-16 修正:此前判定不看状态,解绑微信退回后 0.1 被误判已用、资格永久锁死)。
_NEWBIE_TIER_HELD_STATUSES = {"reviewing", "pending", "success"}
# 免确认收款授权状态
_WX_AUTH_ACTIVE = "TAKING_EFFECT" # 已生效,可免确认转账
_WX_AUTH_CLOSED = "CLOSED" # 已关闭(用户/商户/风控),需重新开启
@@ -374,6 +379,7 @@ def bind_wechat_openid(db: Session, user_id: int, code: str) -> dict:
user.wechat_openid = info["openid"]
user.wechat_nickname = info["nickname"]
user.wechat_avatar_url = info["avatar_url"]
apply_wechat_display_identity(user, wechat_nickname=info["nickname"], wechat_avatar_url=info["avatar_url"])
db.commit()
return info
@@ -622,9 +628,10 @@ def _beijing_today_start_utc() -> datetime:
def withdraw_tier_states(db: Session, user_id: int, source: str = "coin_cash") -> list[dict]:
"""福利页(coin_cash)提现档位的可提现状态。withdraw-info 下发与 create_withdraw 校验共用此口径。
规则(2026-07-09 拍板,7-9提现ui对齐):
- 新人档(0.1/0.3):账号历史一次性只要发起过(**任意状态**,含被拒/失败,"发起就算")
即视为已用,直接**从返回列表消失**;两档各自独立互不影响,不参与"每日选一个额度"互斥
规则(2026-07-09 拍板,7-9提现ui对齐;新人档判定 2026-07-16 修正):
- 新人档(0.1/0.3):账号历史一次性进行中(reviewing/pending)或成功打款(success)即视为
已用,直接**从返回列表消失**;被拒/转账失败/解绑退回(均已退款钱没到手)则恢复可提,不永久
占用资格两档各自独立互不影响,不参与"每日选一个额度"互斥
- 常规档(0.5×3 / 10×1 / 20×1):按北京日计次,"发起就算占用"(当天创建的单不论最终状态
都计入,被拒/失败不退当天名额);三档每天只能选一个,选定后其余两档当天 other_tier_selected
- invite_cash 本轮无档位概念 返回空列表(邀请页客户端仍用本地写死档位,行为不变)
@@ -635,7 +642,7 @@ def withdraw_tier_states(db: Session, user_id: int, source: str = "coin_cash") -
tiers = rewards.WITHDRAW_TIERS_COIN_CASH
amounts = [t.amount_cents for t in tiers]
newbie_amounts = [t.amount_cents for t in tiers if t.is_newbie]
# 新人档历史是否用过:任意时间、任意状态("发起就算")
# 新人档历史是否用过:进行中或已成功打款的单占用资格;被拒/失败/解绑退回(已退款)不算(恢复可提)
used_newbie: set[int] = set(
db.execute(
select(WithdrawOrder.amount_cents)
@@ -644,6 +651,7 @@ def withdraw_tier_states(db: Session, user_id: int, source: str = "coin_cash") -
WithdrawOrder.user_id == user_id,
WithdrawOrder.source == "coin_cash",
WithdrawOrder.amount_cents.in_(newbie_amounts),
WithdrawOrder.status.in_(_NEWBIE_TIER_HELD_STATUSES),
)
).scalars()
) if newbie_amounts else set()
+61
View File
@@ -102,3 +102,64 @@ class RefreshRequest(BaseModel):
class LogoutResponse(BaseModel):
ok: bool = True
# ===== 微信登录 =====
class WechatLoginRequest(BaseModel):
code: str = Field(..., min_length=1, description="微信 App 授权拿到的 code(单次有效)")
device_id: str = Field(
"", max_length=64,
description="硬件级设备标识(Android ANDROID_ID),用于新手引导按 设备+账号 去重;空=按未完成处理",
)
class WechatLoginResponse(BaseModel):
# status="logged_in" → openid 命中,token 有值;"need_bind_phone" → 未命中,bind_ticket 有值
status: str
token: TokenWithUser | None = None
bind_ticket: str | None = None
wechat_nickname: str | None = None
wechat_avatar_url: str | None = None
class OccupiedAccountInfo(BaseModel):
"""手机号被占用时返回的原账号脱敏展示信息(供冲突页)。"""
nickname: str | None = None
avatar_url: str | None = None
created_at: datetime
has_wechat: bool = False
class WechatBindResultResponse(BaseModel):
# status="logged_in" → 未占用,已建号登入,token 有值;
# "phone_occupied" → 手机号被占用,occupied_account + conflict_ticket 有值,token 为 None
status: str
token: TokenWithUser | None = None
occupied_account: OccupiedAccountInfo | None = None
conflict_ticket: str | None = None # 占用时签发,换绑/继续绑定只认它
rebind_available: bool | None = None # 该手机号 30 天内是否还能换绑(给换绑按钮预置禁用态)
rebind_blocked_days: int | None = None # 被限时剩余天数(rebind_available=False 时>0)
class WechatBindPhoneSmsRequest(BaseModel):
bind_ticket: str = Field(..., min_length=1)
phone: str = Field(..., min_length=11, max_length=11, pattern=r"^1\d{10}$")
code: str = Field(..., min_length=4, max_length=8)
device_id: str = Field("", max_length=64)
class WechatBindPhoneJverifyRequest(BaseModel):
bind_ticket: str = Field(..., min_length=1)
login_token: str = Field(..., min_length=1, description="客户端 loginAuth 拿到的 loginToken")
device_id: str = Field("", max_length=64)
class WechatConflictContinueRequest(BaseModel):
conflict_ticket: str = Field(..., min_length=1)
device_id: str = Field("", max_length=64)
class WechatConflictRebindRequest(BaseModel):
conflict_ticket: str = Field(..., min_length=1)
device_id: str = Field("", max_length=64)
-14
View File
@@ -36,20 +36,6 @@ class OkResponse(BaseModel):
ok: bool = True
class PendingCompare(BaseModel):
"""PoC:后端通过心跳下发的"用户要从某源平台比价"信号。前端据此弹选平台窗 + launch 源平台。"""
source_platform: str # pricebot 源平台代号(PoC 写死 "meituan")
source_package: str # 源平台 Android 包名(前端 launch 用)
class HeartbeatResponse(BaseModel):
"""心跳响应。常规只回 ok;PoC 期带回 pending_compare 触发比价(exclude_none 省略 null)。"""
model_config = ConfigDict()
ok: bool = True
pending_compare: PendingCompare | None = None
class LivenessOut(BaseModel):
"""本机掉线告警状态(后置检测 pull 版)。客户端只需这一个布尔判断要不要弹「开启自启动」引导,
故只返回 kill_alert_pending(不暴露设备详情 / 内部 liveness_state )从未注册过 默认 False(无告警)"""
-13
View File
@@ -1,13 +0,0 @@
"""会话存档独立 poller → app-server 内部信号端点的 schema。"""
from __future__ import annotations
from pydantic import BaseModel
class WxFinancePendingIn(BaseModel):
source: str = "meituan" # 源平台代号(pop 后前端 launch 用;PoC 写死 meituan)
kind: str = "" # 触发物类型 weapp / image(仅日志用)
class WxFinancePendingOut(BaseModel):
ok: bool = True
+5 -1
View File
@@ -19,7 +19,11 @@ server {
ssl_ciphers HIGH:!aNULL:!MD5;
ssl_session_cache shared:SSL:10m;
client_max_body_size 4m;
# 上传接口(反馈/上报截图、头像)业务上限 = 最多 6 张 × 每张 5MB
# (见 app _MAX_IMAGES / AVATAR_MAX_BYTES)≈ 30MB,留余量设 32m。
# 低于此值时带截图的反馈会在到达 uvicorn 前就被 nginx 413,表现为「提交经常失败」
# (纯文字反馈体积小、不受影响 → 呈现为「时好时坏」)。根治仍需客户端上传前压缩。
client_max_body_size 32m;
location / {
proxy_pass http://127.0.0.1:8770;
+57
View File
@@ -0,0 +1,57 @@
# OpenObserve 监控台反代(observe.shaguabijia.com)。证书走 Certbot/Let's Encrypt,与 admin-web 一致。
#
# 前置(一次性):
# 1) DNS: observe.shaguabijia.com A 记录 → 本服务器公网 IP
# 2) 证书: sudo certbot certonly --nginx -d observe.shaguabijia.com
# (options-ssl-nginx.conf / ssl-dhparams.pem 首次跑 certbot 时已生成,admin-web 在用即已存在)
# 3) OpenObserve 只绑 127.0.0.1:5080(见 docker-compose.prod.yml),本文件把它反代出公网
# 4) nginx -t 通过后 systemctl reload nginx
#
# 安全:OO 有自身登录。监控台不必对全网裸开——本机办公网无固定出口 IP,故在 nginx 层加 Basic Auth 兜底;
# 将来有固定 IP 可改用【IP 白名单】块(更省事,可去掉 Basic Auth)。
server {
server_name observe.shaguabijia.com;
client_max_body_size 10m;
# —— IP 白名单:办公网无固定出口 IP,暂不用;将来有固定 IP 可改用这块(比 Basic Auth 省事)——
# allow 1.2.3.4; # ← 换成你的真实出口 IP,可多行
# deny all;
# —— Basic Auth:无固定 IP 的兜底密码(生成 .htpasswd_observe 的命令见 README/下方)——
auth_basic "OpenObserve";
auth_basic_user_file /etc/nginx/conf.d/.htpasswd_observe;
location / {
proxy_pass http://127.0.0.1:5080;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# OpenObserve 有实时/流式面板,需透传 WebSocket
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_read_timeout 300s;
}
# IPv6 这行不带 ipv6only=on:该选项对 [::]:443 全局只能设一次,admin-web 那个 server 块已设(否则 nginx 报 duplicate listen options)
listen [::]:443 ssl; # managed by Certbot
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/observe.shaguabijia.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/observe.shaguabijia.com/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
server {
if ($host = observe.shaguabijia.com) {
return 301 https://$host$request_uri;
} # managed by Certbot
listen 80;
listen [::]:80;
server_name observe.shaguabijia.com;
return 404; # managed by Certbot
}
+4
View File
@@ -0,0 +1,4 @@
# OpenObserve 落盘数据(parquet/索引/元数据),运行时产生,不入库。
data/
# 生产 compose 的密码文件(OO_ROOT_PASSWORD),含机密,不入库。
.env
+126
View File
@@ -0,0 +1,126 @@
# OpenObserve 本地部署(接口 QPS / 耗时可观测)
app-server 通过中间件采集每个接口的 QPS + 耗时 + 错误率,批量上报到这里。
设计见 [../../docs/superpowers/specs/2026-07-06-openobserve-api-metrics-design.md](../../docs/superpowers/specs/2026-07-06-openobserve-api-metrics-design.md)。
## 启动
```bash
cd deploy/openobserve
docker compose up -d
```
- Web UIhttp://localhost:5080
- 登录:`admin@shaguabijia.local` / `Complexpass#123`(见 `docker-compose.yml`
- 数据落 `deploy/openobserve/data/`(已挂卷持久化;该目录已 gitignore)
## 让 app-server 上报
在项目根的 `.env` 打开观测(`OBSERVE_*`,账号密码与 compose 里 root 一致):
```dotenv
OBSERVE_ENABLED=true
OBSERVE_ENDPOINT=http://localhost:5080
OBSERVE_ORG=default
OBSERVE_STREAM=app_requests
OBSERVE_USER=admin@shaguabijia.local
OBSERVE_PASSWORD=Complexpass#123
```
重启 app-server,随便打几个接口。stream `app_requests` **首次上报自动创建**
在 UI 的 Logs → 选 `app_requests` 就能看到逐条请求事件(字段:`method` / `route` /
`status` / `duration_ms` / `service` / `env`)。
> 未开 `OBSERVE_ENABLED` 或缺账号密码时,中间件透传、worker 不启动,整套 no-op,不影响业务。
## 查询(Logs 页 SQL,或建 Dashboard 面板)
各接口 QPS(1 分钟分桶,面板里再除 60 得每秒):
```sql
SELECT route, histogram(_timestamp, '1 minute') AS ts, count(*) AS cnt
FROM app_requests GROUP BY route, ts ORDER BY ts
```
各接口 P95 耗时(毫秒):
```sql
SELECT route, approx_percentile_cont(duration_ms, 0.95) AS p95_ms
FROM app_requests GROUP BY route ORDER BY p95_ms DESC
```
各接口错误率(5xx 占比):
```sql
SELECT route,
count(*) FILTER (WHERE status >= 500) * 100.0 / count(*) AS err_pct
FROM app_requests GROUP BY route ORDER BY err_pct DESC
```
## 一键导入现成仪表盘(QPS / P95 / 分位 / 错误率)
备好了 [dashboard-api-metrics.json](dashboard-api-metrics.json),4 个面板:各接口每分钟请求数(QPS 源)、
P95 耗时折线、P50/P95/P99 分位表、5xx 错误率表。
- **UI 导入**Dashboards → 右上 **Import** → 选该 JSON 文件 → Import(每次导入新建,不覆盖)。
- **或 API 导入**
```bash
curl -u admin@shaguabijia.local:Complexpass#123 -H 'Content-Type: application/json' \
-X POST 'http://localhost:5080/api/default/dashboards?folder=default' \
--data-binary @deploy/openobserve/dashboard-api-metrics.json
```
导入后进仪表盘,右上角时间调到「最近 15 分钟 / 1 小时」、开自动刷新即可。低流量下 QPS 面板看「每分钟请求数」比「每秒」直观。
## 停止 / 清数据
```bash
docker compose down # 停止(保留数据)
docker compose down -v && rm -rf data # 停止并清空数据
```
## 生产部署(单机)+ UI 访问
前提:app-server 与 OpenObserve **同机**,app→OO 走 localhost(`127.0.0.1:5080`)、不出网、无需 TLS。
唯一要防的是**别把 :5080 裸暴露公网**。硬化版编排见 [docker-compose.prod.yml](docker-compose.prod.yml)。
### 部署步骤
```bash
# 1) 密码文件(本目录,已 gitignore)
echo "OO_ROOT_PASSWORD=$(python -c 'import secrets;print(secrets.token_urlsafe(24))')" > deploy/openobserve/.env
# 2) 起 OpenObserve(只绑 127.0.0.1、命名卷持久化、mem 1g)
cd deploy/openobserve && docker compose -f docker-compose.prod.yml up -d
sudo systemctl enable docker # 开机自起
```
3) app-server 的 `.env` 打开观测并**重启**(用非 root 的专用 ingest 账号):
```dotenv
OBSERVE_ENABLED=true
OBSERVE_ENDPOINT=http://127.0.0.1:5080
OBSERVE_ORG=default
OBSERVE_STREAM=app_requests
OBSERVE_USER=ingest@shaguabijia.com # UI → Users 建的非 root 账号
OBSERVE_PASSWORD=<该账号密码>
```
```bash
sudo systemctl restart shaguabijia-app-server # 日志出现 "observe worker started" 即生效
```
4) 两个必做收口(磁盘/安全):
- **保留期**:UI → Streams → `app_requests` → Data Retention 设 14/30 天(一请求一行,不封顶迟早撑爆盘)。
- **专用账号**:UI → Users 建非 root 账号给 app 上报,root 只留人工登 UI。
### UI 访问(二选一)
**A. SSH 隧道(推荐,零暴露、不用域名/证书):**
```bash
ssh -L 5080:127.0.0.1:5080 用户@服务器IP
# 然后本机浏览器开 http://localhost:5080
```
**B. nginx 子域名反代(要固定 URL / 团队常看):** 见 [../nginx/observe.shaguabijia.com.conf](../nginx/observe.shaguabijia.com.conf)。
需 DNS `observe.shaguabijia.com` → 本机 + 证书放 `/etc/nginx/ssl/`;含 IP 白名单 + TLS + WebSocket 透传。
> ⚠️ prod compose 必须保持 `127.0.0.1:5080:5080`;写成 `5080:5080`(绑 0.0.0.0)= 裸暴露公网,这是唯一真正的坑。
@@ -0,0 +1,302 @@
{
"version": 8,
"dashboardId": "api-metrics",
"title": "接口监控 (QPS / 耗时 / 错误率)",
"description": "app-server 接口 QPS、P50/P95/P99 耗时、5xx 错误率。数据流 app_requests。",
"role": "",
"tabs": [
{
"tabId": "default",
"name": "Default",
"panels": [
{
"id": "panel_qps",
"type": "line",
"title": "各接口 每分钟请求数 (QPS 源)",
"description": "",
"config": {
"show_legends": true,
"legends_position": null,
"decimals": 2.0,
"axis_border_show": false,
"base_map": null,
"map_view": null
},
"queryType": "sql",
"queries": [
{
"query": "SELECT histogram(_timestamp, '1 minute') as ts, route, count(*) as reqs FROM app_requests GROUP BY ts, route ORDER BY ts",
"vrlFunctionQuery": "",
"customQuery": true,
"fields": {
"stream": "app_requests",
"stream_type": "logs",
"x": [
{
"label": "ts",
"alias": "ts",
"column": "ts",
"color": null,
"sortBy": "ASC"
}
],
"y": [
{
"label": "reqs",
"alias": "reqs",
"column": "reqs",
"color": null
}
],
"z": [],
"breakdown": [
{
"label": "route",
"alias": "route",
"column": "route",
"color": null
}
],
"filter": {
"filterType": "group",
"logicalOperator": "AND",
"conditions": []
}
},
"config": {
"promql_legend": "",
"layer_type": "scatter",
"weight_fixed": 1.0
}
}
],
"layout": {
"x": 0,
"y": 0,
"w": 24,
"h": 9,
"i": 1
}
},
{
"id": "panel_p95",
"type": "line",
"title": "各接口 P95 耗时 (ms)",
"description": "",
"config": {
"show_legends": true,
"legends_position": null,
"decimals": 2.0,
"axis_border_show": false,
"base_map": null,
"map_view": null
},
"queryType": "sql",
"queries": [
{
"query": "SELECT histogram(_timestamp, '1 minute') as ts, route, approx_percentile_cont(duration_ms, 0.95) as p95_ms FROM app_requests GROUP BY ts, route ORDER BY ts",
"vrlFunctionQuery": "",
"customQuery": true,
"fields": {
"stream": "app_requests",
"stream_type": "logs",
"x": [
{
"label": "ts",
"alias": "ts",
"column": "ts",
"color": null,
"sortBy": "ASC"
}
],
"y": [
{
"label": "p95_ms",
"alias": "p95_ms",
"column": "p95_ms",
"color": null
}
],
"z": [],
"breakdown": [
{
"label": "route",
"alias": "route",
"column": "route",
"color": null
}
],
"filter": {
"filterType": "group",
"logicalOperator": "AND",
"conditions": []
}
},
"config": {
"promql_legend": "",
"layer_type": "scatter",
"weight_fixed": 1.0
}
}
],
"layout": {
"x": 24,
"y": 0,
"w": 24,
"h": 9,
"i": 2
}
},
{
"id": "panel_pctl",
"type": "table",
"title": "各接口 耗时分位 P50/P95/P99 (ms)",
"description": "",
"config": {
"show_legends": true,
"legends_position": null,
"decimals": 2.0,
"axis_border_show": false,
"base_map": null,
"map_view": null
},
"queryType": "sql",
"queries": [
{
"query": "SELECT route, approx_percentile_cont(duration_ms,0.5) as p50, approx_percentile_cont(duration_ms,0.95) as p95, approx_percentile_cont(duration_ms,0.99) as p99, count(*) as cnt FROM app_requests GROUP BY route ORDER BY p95 DESC",
"vrlFunctionQuery": "",
"customQuery": true,
"fields": {
"stream": "app_requests",
"stream_type": "logs",
"x": [
{
"label": "route",
"alias": "route",
"column": "route",
"color": null
}
],
"y": [
{
"label": "p50",
"alias": "p50",
"column": "p50",
"color": null
},
{
"label": "p95",
"alias": "p95",
"column": "p95",
"color": null
},
{
"label": "p99",
"alias": "p99",
"column": "p99",
"color": null
},
{
"label": "cnt",
"alias": "cnt",
"column": "cnt",
"color": null
}
],
"z": [],
"breakdown": [],
"filter": {
"filterType": "group",
"logicalOperator": "AND",
"conditions": []
}
},
"config": {
"promql_legend": "",
"layer_type": "scatter",
"weight_fixed": 1.0
}
}
],
"layout": {
"x": 0,
"y": 9,
"w": 24,
"h": 9,
"i": 3
}
},
{
"id": "panel_err",
"type": "table",
"title": "各接口 错误率 (5xx %)",
"description": "",
"config": {
"show_legends": true,
"legends_position": null,
"decimals": 2.0,
"axis_border_show": false,
"base_map": null,
"map_view": null
},
"queryType": "sql",
"queries": [
{
"query": "SELECT route, count(*) FILTER (WHERE status >= 500) * 100.0 / count(*) as err_pct, count(*) as cnt FROM app_requests GROUP BY route ORDER BY err_pct DESC",
"vrlFunctionQuery": "",
"customQuery": true,
"fields": {
"stream": "app_requests",
"stream_type": "logs",
"x": [
{
"label": "route",
"alias": "route",
"column": "route",
"color": null
}
],
"y": [
{
"label": "err_pct",
"alias": "err_pct",
"column": "err_pct",
"color": null
},
{
"label": "cnt",
"alias": "cnt",
"column": "cnt",
"color": null
}
],
"z": [],
"breakdown": [],
"filter": {
"filterType": "group",
"logicalOperator": "AND",
"conditions": []
}
},
"config": {
"promql_legend": "",
"layer_type": "scatter",
"weight_fixed": 1.0
}
}
],
"layout": {
"x": 24,
"y": 9,
"w": 24,
"h": 9,
"i": 4
}
}
]
}
],
"variables": {
"list": []
}
}
@@ -0,0 +1,33 @@
# 生产用 OpenObserve(单机)。相对本地版 docker-compose.yml 的区别:
# - 端口只绑 127.0.0.1 → 公网/外网都到不了(UI 访问走 SSH 隧道或 nginx 反代,见 README)
# - root 密码走环境变量(放同目录 .env,已 gitignore,勿提交)
# - 数据 bind-mount 到宿主 /data 分区(需预建目录 + 确认容器可写)+ CPU/内存上限(与 app/PG 共存防抢内存)
#
# 用法:
# 1) 本目录建 .env(已 gitignore):
# OO_ROOT_PASSWORD=<强随机串> # 生成: python -c "import secrets;print(secrets.token_urlsafe(24))"
# 2) docker compose -f docker-compose.prod.yml up -d
# 3) 开机自起: sudo systemctl enable docker
services:
openobserve:
image: public.ecr.aws/zinclabs/openobserve:v0.91.2
container_name: openobserve
ports:
- "127.0.0.1:5080:5080" # 只绑本机,安全
environment:
ZO_ROOT_USER_EMAIL: "admin@shaguabijia.com"
ZO_ROOT_USER_PASSWORD: "${OO_ROOT_PASSWORD:?请先在 deploy/openobserve/.env 里设 OO_ROOT_PASSWORD}"
ZO_DATA_DIR: "/data"
ZO_COMPACT_DATA_RETENTION_DAYS: "30" # 超 30 天自动删,防爆盘(默认 3650 天=10年)
ZO_TELEMETRY: "false" # 关匿名遥测(内网自用);变量名是 ZO_TELEMETRY,不是 *_ENABLED
volumes:
- /data/openobserve/data:/data # 绑定挂载到宿主机的 /data/openobserve/data 目录(建议该目录所在分区有 20G+ 空间)
restart: unless-stopped
deploy:
resources:
limits: # 硬上限:防 OO 查询/ingest 抢爆 CPU/内存,拖垮同机 PG+app
cpus: '2.0'
memory: 3G
logging: # 容器 stdout 日志上限,防爆盘
driver: json-file
options: { max-size: "10m", max-file: "3" }
+16
View File
@@ -0,0 +1,16 @@
# 本地开发用 OpenObserve(单容器 = local 模式)。用于接收 app-server 的接口指标(QPS/耗时/错误率)。
# 启动: cd deploy/openobserve && docker compose up -d
# Web UI: http://localhost:5080 (账号见下方 env)
services:
openobserve:
image: public.ecr.aws/zinclabs/openobserve:latest
container_name: openobserve
ports:
- "5080:5080"
environment:
ZO_ROOT_USER_EMAIL: "admin@shaguabijia.local"
ZO_ROOT_USER_PASSWORD: "Complexpass#123"
ZO_DATA_DIR: "/data"
volumes:
- ./data:/data
restart: unless-stopped
-18
View File
@@ -1,18 +0,0 @@
[Unit]
# 会话存档轮询 poller —— 独立进程跑 WeWorkFinanceSdk(Go c-shared .so),与 app-server 隔离:
# 嵌进 app-server 会 segfault 把主进程带崩,拆出来后崩了 systemd 只重启本服务、不影响 app-server。
Description=Shaguabijia 会话存档轮询 poller (WeWorkFinanceSdk, isolated from app-server)
After=network.target shaguabijia-app-server.service
[Service]
Type=simple
User=root
WorkingDirectory=/opt/shaguabijia-app-server
Environment="PATH=/opt/shaguabijia-app-server/.venv/bin:/usr/bin:/bin"
EnvironmentFile=/opt/shaguabijia-app-server/.env
ExecStart=/opt/shaguabijia-app-server/.venv/bin/python scripts/wx_finance_poller.py
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
+2
View File
@@ -42,6 +42,8 @@
| `ad_ecpm_record` | 广告展示 eCPM 上报(收益对账) | `models/ad_ecpm.py` | [详情](./ad_ecpm_record.md) |
| `ad_feed_reward_record` | 信息流/Draw 广告结算记录(10 秒一份,client_event_id 幂等;`ad_type`+`feed_scene` 分形态/场景) | `models/ad_feed_reward.py` | [详情](./ad_feed_reward_record.md) |
| `ad_pangle_daily_revenue` | 穿山甲 GroMore 后台收益日表(定时拉取,收益报表/大盘真实收益源,#92) | `models/ad_pangle_revenue.py` | [详情](./ad_pangle_daily_revenue.md) |
| `inactivity_reset_log` | 15 天不活跃清零审计(每次清零一行;清零前三桶余额快照+原因+不活跃天数;只清金币+现金,邀请金仅快照) | `models/inactivity.py` | [详情](./inactivity_reset_log.md) |
| `inactivity_notification_log` | 不活跃清零前预警记录(余额快照+档位+通道+状态;streak 去重依据 + 占位 outbox) | `models/inactivity.py` | [详情](./inactivity_notification_log.md) |
### 比价 / 省钱
| 表 | 用途 | 模型 | 文档 |
@@ -0,0 +1,36 @@
# inactivity_notification_log — 不活跃清零前预警记录
> 模型 `app/models/inactivity.py` · 仓库 `app/repositories/inactivity.py` · 通知器 `app/integrations/notifier.py` · [← 索引](./README.md) · [总览](./OVERVIEW.md)
清零前按可配置节奏(`INACTIVITY_WARN_DAYS_BEFORE`,默认清零前 7 天、2 天各一次)向用户预警"账户里的 xx 金币和 xx 现金将被清零"。每发一次预警写一行,记推送时的余额快照 + 提前天数档 + 通道 + 状态。兼作两用:**预警去重**依据(同 streak 内 `stage==k 且 created_at > last_active` 即已推过、不重推)与**占位 outbox**(v1 通道=`log`,只打日志不真推;后续接 JPush/短信同层扩展)。append-only,不更新。**预警只涉及会被清的金币 + 折算现金;邀请奖励金不清、不预警**(`invite_cash_balance_cents` 仅作账户状态快照)。
## 用在哪 / 增删改查
- **C(插入)**:`inactivity.run_warn_once` 命中预警档、且本 streak 未推过时,调 `notifier.warn` 后写一行(`status` = 通知器返回,占位实现为 `placeholder`)。
- **U / D**:无(append-only)。
- **R**:预警去重查询(`user_id + stage + created_at > last_active`);未来接真实推送时作待推送 outbox。
## 字段
| 列 | 类型 | 约束 / 默认 | 说明(取值 / join) |
|---|---|---|---|
| `id` | Integer | **PK**, autoincrement | 主键 |
| `user_id` | Integer | NOT NULL, index | 预警对象;只索引不设外键(同 `analytics_event`) |
| `stage` | Integer | NOT NULL | 提前天数档(如 `7` / `2`,即清零前第几天推) |
| `inactive_days` | Integer | NOT NULL | 推送时的不活跃天数(北京自然日) |
| `coin_balance` | Integer | NOT NULL | 推送时金币余额快照(将被清) |
| `cash_balance_cents` | Integer | NOT NULL | 推送时折算现金余额快照(分,将被清) |
| `invite_cash_balance_cents` | Integer | NOT NULL | 推送时**邀请奖励金**余额快照(分,**不清、不在预警额度内**) |
| `channel` | String(16) | NOT NULL | 通道:`log`(占位) / `jpush` / `sms` |
| `status` | String(16) | NOT NULL | 状态:`placeholder`(占位未真推) / `sent` / `failed` |
| `created_at` | DateTime(tz) | server_default now(), index | 推送时刻;去重比 `created_at > last_active`(用户回归后 `last_active` 前移 → 旧行自然失效、开启新 streak) |
## 关系 / Join Key
- `user_id``user.id`(无外键直连,靠 `user_id` 关联)。
- 与 `inactivity_reset_log` 无直接外键;同一 streak 内先有若干预警行,到期后有一行清零。
## 索引与约束
- PK `id`;`ix_inactivity_notification_log_user_id``ix_inactivity_notification_log_created_at`
## 注意
- **预警去重按 streak**:判据是 `created_at > last_active`;用户一有活跃(`home_view`/比价/领券),`last_active` 前移,旧预警行"失效",回归后可重新进入预警。
- **占位实现**:v1 `LogNotifier``logger.warning("[inactivity-warn] ...")`、返回 `placeholder`,不真推(参照心跳告警"本期先不接推送"先例)。
- **漏跑补发**:worker 漏跑数天后某用户可能同时满足多档,只补发**最紧急的未推档**(最小提前天数),避免刷屏。
+35
View File
@@ -0,0 +1,35 @@
# inactivity_reset_log — 15 天不活跃清零审计
> 模型 `app/models/inactivity.py` · 仓库 `app/repositories/inactivity.py` · worker `app/core/inactivity_reset_worker.py` · [← 索引](./README.md) · [总览](./OVERVIEW.md)
连续 15 天不活跃(北京自然日,活跃口径见 `app/repositories/activity.py`:`home_view` + 发起比价 + 发起领券,**不含登录**)的用户,worker 每日自动清零其**金币 + 折算现金**。每清一个用户写一行,记清零前三桶余额快照 + 原因 + 判定时的活跃时间/不活跃天数,供纠纷排查。清零同时另写 2 条钱包流水(`coin_transaction` / `cash_transaction`,`biz_type=inactivity_reset`,`ref_id=` 本表 `id`),资金流可逐笔回溯、人工恢复。**邀请奖励金(`invite_cash_balance_cents`)是产品红线、不清零**,本表 `invite_cash_balance_cents_before` 仅为清零时仍保留的邀请金快照(非被清金额)。append-only,不更新。
## 用在哪 / 增删改查
- **C(插入)**:`inactivity.clear_user` 逐用户清零(独立事务、行锁)时写一行,`db.flush()``id` 作流水 `ref_id` 交叉链接。
- **U / D**:无(append-only 审计)。
- **R**:纠纷排查 / 对账(与 `coin_transaction` / `cash_transaction``ref_id` 交叉核对)。
## 字段
| 列 | 类型 | 约束 / 默认 | 说明(取值 / join) |
|---|---|---|---|
| `id` | Integer | **PK**, autoincrement | 主键;作 `ref_id` 写入两条清零流水 |
| `user_id` | Integer | NOT NULL, index | 被清零用户;只索引不设外键(同 `analytics_event`,避免删用户级联 / 留历史) |
| `coin_balance_before` | Integer | NOT NULL | 清零前金币余额(个数);= 对应 `coin_transaction.amount` 绝对值 |
| `cash_balance_cents_before` | Integer | NOT NULL | 清零前折算现金余额(分);= 对应 `cash_transaction.amount_cents` 绝对值 |
| `invite_cash_balance_cents_before` | Integer | NOT NULL | 清零时的**邀请奖励金**余额快照(分)——**不清、原封保留**,仅记录以证明"未动邀请金" |
| `last_active_at` | DateTime(tz) | nullable | 判定时的最近活跃时刻(UTC);无任何活跃信号时兜底为 `user.created_at` |
| `inactive_days` | Integer | NOT NULL | 判定时的不活跃天数(北京自然日) |
| `reason` | String(32) | NOT NULL | 清零原因,如 `inactive_15d` |
| `reset_at` | DateTime(tz) | server_default now(), index | 清零时刻 |
## 关系 / Join Key
- `user_id``user.id`(无外键直连,靠 `user_id` 关联)。
- `id``coin_transaction.ref_id` / `cash_transaction.ref_id`(`biz_type=inactivity_reset`):审计行 ↔ 资金流水交叉对账。
## 索引与约束
- PK `id`;`ix_inactivity_reset_log_user_id`(按用户查)、`ix_inactivity_reset_log_reset_at`(按时间查)。
## 注意
- **只清 2 桶**:金币 + 折算现金;**邀请现金不清**(两本账物理隔离,见 [`coin_account`](./coin_account.md) / `wallet.CoinAccount` 注释)。
- **天然幂等**:清完余额=0,次日不再匹配;worker 重启 / 多次唤醒 / 补跑都不会重复清零或重复流水。
- **总闸默认关**(`INACTIVITY_RESET_ENABLED=false`),灰度验证清零名单后再开。
@@ -0,0 +1,897 @@
# 接口 QPS + 耗时可观测(OpenObserve)实现计划
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** 给 app-server 每个接口采集 QPS + 耗时 + 错误率,经轻量 ASGI 中间件 + 后台 worker 批量直采到本地 Docker 的 OpenObserve。
**Architecture:** 纯 ASGI 中间件测每请求耗时/抓路由模板+状态码 → 非阻塞入有界队列(满则丢、绝不阻塞)→ 后台 asyncio worker 批量 POST 到 OpenObserve `_json` ingest 端点。请求路径零 I/O;未配置观测则整套 no-op;上报失败丢批不重试。
**Tech Stack:** FastAPI / Starlette ASGI 中间件、`asyncio.Queue``httpx.AsyncClient`(已有依赖)、pydantic-settings、OpenObserveDocker)。
参考 spec[docs/superpowers/specs/2026-07-06-openobserve-api-metrics-design.md](2026-07-06-openobserve-api-metrics-design.md)
---
## 文件结构
| 文件 | 职责 |
|---|---|
| `app/core/config.py`(改) | 新增 `OBSERVE_*` 配置 + `observe_configured` 门槛属性 |
| `app/core/observe.py`(新) | 有界事件队列 + `record_event` + 路由模板解析 + `RequestMetricsMiddleware` |
| `app/core/observe_worker.py`(新) | 后台批量上报 worker:`_collect_batch` / `_post_batch` / `start_*` / `stop_*` |
| `app/main.py`(改) | 挂中间件(最外层)+ lifespan 启停 worker |
| `.env.example`(改) | 新增 `OBSERVE_*` 注释段 |
| `deploy/openobserve/docker-compose.yml`(新) | 本地 OpenObserve 容器 |
| `deploy/openobserve/README.md`(新) | 部署步骤 + 查询/仪表盘 SQL |
| `tests/test_observe.py`(新) | 配置门槛 / 队列 / 中间件 / worker 单测 |
**关键接口契约(跨任务一致,勿改名):**
- `app.core.observe.get_queue() -> asyncio.Queue[dict]`
- `app.core.observe.record_event(event: dict) -> None`
- `app.core.observe.take_dropped() -> int`
- `app.core.observe.RequestMetricsMiddleware`ASGI class`__init__(self, app)`
- 事件字段:`_timestamp`(µs int) / `service` / `env` / `method` / `route` / `status` / `duration_ms`(float)
- `app.core.observe_worker.start_observe_worker() -> asyncio.Task | None`
- `app.core.observe_worker.stop_observe_worker(task) -> None`
- `settings.observe_configured -> bool`
---
## Task 1: 配置项 `OBSERVE_*` + `observe_configured`
**Files:**
- Modify: `app/core/config.py`(在 `cors_origins_list` property 之后、`is_prod` property 之前插入)
- Test: `tests/test_observe.py`(新建)
- [ ] **Step 1: 写失败测试**
新建 `tests/test_observe.py`
```python
"""接口指标可观测(observe)单测:配置门槛 / 队列 / 中间件 / worker。
沿用仓库约定:TestClient + monkeypatch,绝不打真网络。observe 默认关(conftest 未设
OBSERVE_*),需要开启的用例用 monkeypatch 改 settings 单例属性。
"""
from __future__ import annotations
from app.core.config import settings
def test_observe_configured_requires_switch_and_creds(monkeypatch):
# 开关开 + endpoint(默认 localhost)+ user + password 齐全 → True
monkeypatch.setattr(settings, "OBSERVE_ENABLED", True)
monkeypatch.setattr(settings, "OBSERVE_USER", "u")
monkeypatch.setattr(settings, "OBSERVE_PASSWORD", "p")
assert settings.observe_configured is True
# 缺密码 → False
monkeypatch.setattr(settings, "OBSERVE_PASSWORD", "")
assert settings.observe_configured is False
# 开关关 → False(即便凭证齐全)
monkeypatch.setattr(settings, "OBSERVE_PASSWORD", "p")
monkeypatch.setattr(settings, "OBSERVE_ENABLED", False)
assert settings.observe_configured is False
```
- [ ] **Step 2: 运行,确认失败**
Run: `pytest tests/test_observe.py::test_observe_configured_requires_switch_and_creds -q`
Expected: FAIL —— `AttributeError``settings``OBSERVE_ENABLED` / 无 `observe_configured`
- [ ] **Step 3: 实现配置**
`app/core/config.py``cors_origins_list` property 之后、`is_prod` property 之前插入:
```python
# ===== 可观测(OpenObserve 接口指标)=====
# 采集每个接口的 QPS + 耗时 + 错误率,批量直采到 OpenObserve(本地 Docker)。
# 默认关(prod 安全):未开启 → 中间件透传、worker 不启动,整套 no-op。
# 开启需 ENABLED=true 且 ENDPOINT/USER/PASSWORD 齐全(见 observe_configured)。
OBSERVE_ENABLED: bool = False
OBSERVE_ENDPOINT: str = "http://localhost:5080" # OpenObserve base URL
OBSERVE_ORG: str = "default" # 组织名
OBSERVE_STREAM: str = "app_requests" # stream 名(首次上报自动建)
OBSERVE_USER: str = "" # Basic auth 邮箱
OBSERVE_PASSWORD: str = "" # Basic auth 密码/token
OBSERVE_FLUSH_INTERVAL_SEC: float = 5.0 # worker 最长攒批间隔
OBSERVE_BATCH_MAX: int = 200 # 单批最大事件数
OBSERVE_QUEUE_MAX: int = 10000 # 有界队列上限,满则丢
OBSERVE_TIMEOUT_SEC: float = 5.0 # 上报 HTTP 超时
@property
def observe_configured(self) -> bool:
"""观测上报可用 = 总开关开 且 endpoint/账号/密码齐全(缺则整套 no-op)。"""
return bool(
self.OBSERVE_ENABLED
and self.OBSERVE_ENDPOINT
and self.OBSERVE_USER
and self.OBSERVE_PASSWORD
)
```
- [ ] **Step 4: 运行,确认通过**
Run: `pytest tests/test_observe.py::test_observe_configured_requires_switch_and_creds -q`
Expected: PASS
- [ ] **Step 5: 提交**
```bash
git add app/core/config.py tests/test_observe.py
git commit -m "feat(observe): 加 OBSERVE_* 配置与 observe_configured 门槛"
```
---
## Task 2: 事件队列 + `record_event` + `take_dropped`
**Files:**
- Create: `app/core/observe.py`
- Test: `tests/test_observe.py`(追加)
- [ ] **Step 1: 写失败测试**
`tests/test_observe.py` 顶部 import 区补 `import asyncio``from app.core import observe`,并追加:
```python
def test_record_event_enqueues(monkeypatch):
q = asyncio.Queue(maxsize=10)
monkeypatch.setattr(observe, "_queue", q)
observe.record_event({"route": "/x"})
assert q.get_nowait() == {"route": "/x"}
def test_record_event_drops_when_full(monkeypatch):
q = asyncio.Queue(maxsize=1)
monkeypatch.setattr(observe, "_queue", q)
monkeypatch.setattr(observe, "_dropped", 0)
observe.record_event({"n": 1}) # 占满
observe.record_event({"n": 2}) # 满 → 丢弃当前,不抛异常
assert observe.take_dropped() == 1
assert observe.take_dropped() == 0 # 取出后清零
assert q.get_nowait() == {"n": 1} # 保留的是先到的
```
- [ ] **Step 2: 运行,确认失败**
Run: `pytest tests/test_observe.py -q -k record_event`
Expected: FAIL —— `ModuleNotFoundError: app.core.observe` 或无 `record_event`
- [ ] **Step 3: 实现 `app/core/observe.py`(先只放队列部分)**
> 注意:本步只放队列相关代码。中间件用到的 `os`/`time`/`Match``_SKIP_PATHS`/`_UNMATCHED`/`_SERVICE` 常量放到 Task 3 一并加入——否则本步提交时 ruff 会报 F401 未用导入。
新建 `app/core/observe.py`
```python
"""接口指标埋点:有界事件队列 + 纯 ASGI 中间件。
每个 HTTP 请求测总耗时、抓路由模板 + 状态码,非阻塞塞进有界队列;由 observe_worker
后台批量上报到 OpenObserve。请求路径上无任何 I/O。未配置观测时中间件直接透传。
"""
from __future__ import annotations
import asyncio
from app.core.config import settings
# 有界事件队列(懒创建,见 get_queue):首次取用时在运行中的 loop 里建,避免 import 期
# 无 loop 的边角问题;put_nowait/get_nowait 不需运行中的 loop → 可在无 loop 下测试。
_queue: asyncio.Queue[dict] | None = None
# 队列满时的丢弃计数,worker 定期取出打日志。
_dropped = 0
def get_queue() -> asyncio.Queue[dict]:
"""返回全局有界事件队列(懒创建)。测试可 monkeypatch 模块级 _queue 换成小队列。"""
global _queue
if _queue is None:
_queue = asyncio.Queue(maxsize=settings.OBSERVE_QUEUE_MAX)
return _queue
def take_dropped() -> int:
"""取出并清零累计丢弃数(供 worker 打点)。"""
global _dropped
n, _dropped = _dropped, 0
return n
def record_event(event: dict) -> None:
"""非阻塞入队;队列满则丢弃当前事件并计数。永不抛异常、永不阻塞请求。"""
global _dropped
try:
get_queue().put_nowait(event)
except asyncio.QueueFull:
_dropped += 1
```
- [ ] **Step 4: 运行,确认通过**
Run: `pytest tests/test_observe.py -q -k record_event`
Expected: PASS
- [ ] **Step 5: 提交**
```bash
git add app/core/observe.py tests/test_observe.py
git commit -m "feat(observe): 加有界事件队列与 record_event(满则丢)"
```
---
## Task 3: `RequestMetricsMiddleware`(路由模板 + 状态码 + 耗时)
**Files:**
- Modify: `app/core/observe.py`(追加 `_resolve_route` 和中间件 class
- Test: `tests/test_observe.py`(追加)
- [ ] **Step 1: 写失败测试**
`tests/test_observe.py` 顶部 import 区补:
```python
import pytest
from fastapi import FastAPI
from fastapi.testclient import TestClient
```
并追加:
```python
def _make_probe_app() -> FastAPI:
"""独立最小 app:只挂中间件 + 两个无鉴权路由,不碰真业务 DB/auth。"""
app = FastAPI()
app.add_middleware(observe.RequestMetricsMiddleware)
@app.get("/things/{tid}")
def get_thing(tid: str):
return {"tid": tid}
@app.get("/health")
def health():
return {"ok": True}
return app
@pytest.fixture
def observe_on(monkeypatch):
"""开启观测 + 换一个干净小队列,返回该队列供断言。"""
monkeypatch.setattr(settings, "OBSERVE_ENABLED", True)
monkeypatch.setattr(settings, "OBSERVE_USER", "u")
monkeypatch.setattr(settings, "OBSERVE_PASSWORD", "p")
q = asyncio.Queue(maxsize=100)
monkeypatch.setattr(observe, "_queue", q)
return q
def test_middleware_records_route_template(observe_on):
client = TestClient(_make_probe_app())
r = client.get("/things/42")
assert r.status_code == 200
evt = observe_on.get_nowait()
assert evt["route"] == "/things/{tid}" # 模板,不是 /things/42
assert evt["method"] == "GET"
assert evt["status"] == 200
assert evt["duration_ms"] >= 0
assert evt["service"] and "env" in evt and isinstance(evt["_timestamp"], int)
def test_middleware_skips_health(observe_on):
client = TestClient(_make_probe_app())
client.get("/health")
assert observe_on.empty()
def test_middleware_unmatched_route_is_normalized(observe_on):
client = TestClient(_make_probe_app())
r = client.get("/definitely-not-a-route")
assert r.status_code == 404
evt = observe_on.get_nowait()
assert evt["route"] == "__unmatched__"
assert evt["status"] == 404
def test_middleware_noop_when_disabled(monkeypatch):
monkeypatch.setattr(settings, "OBSERVE_ENABLED", False)
q = asyncio.Queue(maxsize=100)
monkeypatch.setattr(observe, "_queue", q)
client = TestClient(_make_probe_app())
client.get("/things/1")
assert q.empty() # 未配置观测 → 零入队
```
- [ ] **Step 2: 运行,确认失败**
Run: `pytest tests/test_observe.py -q -k middleware`
Expected: FAIL —— `AttributeError: module 'app.core.observe' has no attribute 'RequestMetricsMiddleware'`
- [ ] **Step 3a: 给 `app/core/observe.py` 补中间件用的导入与常量**
把顶部 import 段从
```python
from __future__ import annotations
import asyncio
from app.core.config import settings
```
改成
```python
from __future__ import annotations
import asyncio
import os
import time
from starlette.routing import Match
from app.core.config import settings
# 不采集的路径(纯噪音):健康检查。
_SKIP_PATHS = frozenset({"/health"})
# 未匹配路由(404/扫描器)归一到此,防维度爆炸。
_UNMATCHED = "__unmatched__"
# service 字段:与 logging.py 同源(LOG_SERVICE_NAME),默认 app-server。
_SERVICE = os.getenv("LOG_SERVICE_NAME", "app-server")
```
`_queue` / `_dropped` / `get_queue` / `take_dropped` / `record_event` 保持不动。)
- [ ] **Step 3b: 实现中间件(追加到 `app/core/observe.py` 末尾)**
```python
def _resolve_route(scope) -> str:
"""从 scope 取路由模板(如 /things/{tid})。优先 scope['route'](现代 Starlette
路由后写入);取不到则手动匹配一次(老版本兜底);仍无 → __unmatched__(404/扫描器)。"""
route = scope.get("route")
path = getattr(route, "path", None)
if path:
return path
app_ = scope.get("app")
router = getattr(app_, "router", None)
for candidate in getattr(router, "routes", []):
try:
match, _ = candidate.matches(scope)
except Exception: # noqa: BLE001 - 匹配兜底,任一路由异常不影响整体
continue
if match == Match.FULL and getattr(candidate, "path", None):
return candidate.path
return _UNMATCHED
class RequestMetricsMiddleware:
"""纯 ASGI 中间件:测每个 http 请求耗时,记 method/route/status/duration。
放在最外层(main.py 里 CORS 之后 add),测到含 CORS 的完整耗时。未配置观测 → 透传。
"""
def __init__(self, app) -> None:
self.app = app
async def __call__(self, scope, receive, send) -> None:
if scope["type"] != "http" or not settings.observe_configured:
await self.app(scope, receive, send)
return
if scope.get("path") in _SKIP_PATHS:
await self.app(scope, receive, send)
return
start = time.perf_counter()
status_holder = {"status": 500} # 下游异常未产出 response 时兜底 500
async def send_wrapper(message) -> None:
if message["type"] == "http.response.start":
status_holder["status"] = message["status"]
await send(message)
try:
await self.app(scope, receive, send_wrapper)
finally:
duration_ms = (time.perf_counter() - start) * 1000.0
record_event({
"_timestamp": int(time.time() * 1_000_000), # µs,OpenObserve 时间列
"service": _SERVICE,
"env": settings.APP_ENV,
"method": scope.get("method", ""),
"route": _resolve_route(scope),
"status": status_holder["status"],
"duration_ms": round(duration_ms, 3),
})
```
- [ ] **Step 4: 运行,确认通过**
Run: `pytest tests/test_observe.py -q -k middleware`
Expected: PASS4 个中间件用例全过)
> 若 `test_middleware_records_route_template` 拿到的是 `/things/42` 而非模板,说明该 Starlette 版本未在 `scope["route"]` 写模板——此时 `_resolve_route` 的手动匹配兜底应已生效并返回模板;若仍不对,检查兜底分支是否被 import 顺序影响。
- [ ] **Step 5: 提交**
```bash
git add app/core/observe.py tests/test_observe.py
git commit -m "feat(observe): 加 RequestMetricsMiddleware(路由模板+状态码+耗时)"
```
---
## Task 4: 后台上报 worker
**Files:**
- Create: `app/core/observe_worker.py`
- Test: `tests/test_observe.py`(追加)
- [ ] **Step 1: 写失败测试**
`tests/test_observe.py` 顶部 import 区补:
```python
import httpx
from app.core import observe_worker
```
并追加:
```python
async def test_collect_batch_drains_up_to_batch_max(monkeypatch):
q = asyncio.Queue(maxsize=100)
monkeypatch.setattr(observe, "_queue", q)
monkeypatch.setattr(settings, "OBSERVE_FLUSH_INTERVAL_SEC", 0.1)
monkeypatch.setattr(settings, "OBSERVE_BATCH_MAX", 200)
for i in range(3):
q.put_nowait({"n": i})
batch = await observe_worker._collect_batch()
assert [e["n"] for e in batch] == [0, 1, 2]
async def test_collect_batch_timeout_returns_empty(monkeypatch):
q = asyncio.Queue(maxsize=100)
monkeypatch.setattr(observe, "_queue", q)
monkeypatch.setattr(settings, "OBSERVE_FLUSH_INTERVAL_SEC", 0.05)
batch = await observe_worker._collect_batch()
assert batch == []
async def test_post_batch_hits_json_ingest_url(monkeypatch):
monkeypatch.setattr(settings, "OBSERVE_ORG", "default")
monkeypatch.setattr(settings, "OBSERVE_STREAM", "app_requests")
captured = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["url"] = str(request.url)
captured["json"] = request.content
return httpx.Response(200, json={"code": 200})
client = httpx.AsyncClient(
base_url="http://oo", transport=httpx.MockTransport(handler)
)
await observe_worker._post_batch(client, [{"route": "/x", "status": 200}])
await client.aclose()
assert captured["url"] == "http://oo/api/default/app_requests/_json"
assert b"/x" in captured["json"]
def test_start_observe_worker_noop_when_not_configured(monkeypatch):
monkeypatch.setattr(settings, "OBSERVE_ENABLED", False)
assert observe_worker.start_observe_worker() is None
```
- [ ] **Step 2: 运行,确认失败**
Run: `pytest tests/test_observe.py -q -k "collect_batch or post_batch or start_observe"`
Expected: FAIL —— `ModuleNotFoundError: app.core.observe_worker`
- [ ] **Step 3: 实现 `app/core/observe_worker.py`**
新建 `app/core/observe_worker.py`
```python
"""接口指标后台上报 worker:批量 drain 事件队列 → POST 到 OpenObserve。
对齐 heartbeat_monitor_worker 等的 start_*/stop_* 形态。best-effort 遥测:catch 全部
异常,上报失败直接丢批不重试。未配置观测 → start 返回 None(不启动),整套 no-op。
"""
from __future__ import annotations
import asyncio
import contextlib
import logging
import httpx
from app.core.config import settings
from app.core.observe import get_queue, take_dropped
logger = logging.getLogger("shagua.observe")
# 上报用的 httpx client,start 时建、stop 时关。
_client: httpx.AsyncClient | None = None
async def _collect_batch() -> list[dict]:
"""等到 ≥1 条(或到 flush 间隔)后,连抽到 BATCH_MAX 条或抽空。超时且空 → 返回 []。"""
queue = get_queue()
batch: list[dict] = []
try:
first = await asyncio.wait_for(
queue.get(), timeout=settings.OBSERVE_FLUSH_INTERVAL_SEC
)
except asyncio.TimeoutError:
return batch
batch.append(first)
while len(batch) < settings.OBSERVE_BATCH_MAX:
try:
batch.append(queue.get_nowait())
except asyncio.QueueEmpty:
break
return batch
async def _post_batch(client: httpx.AsyncClient, batch: list[dict]) -> None:
"""POST 一批事件到 OpenObserve 的 _json ingest 端点。非 2xx 仅告警。"""
url = f"/api/{settings.OBSERVE_ORG}/{settings.OBSERVE_STREAM}/_json"
resp = await client.post(url, json=batch)
if resp.status_code >= 300:
logger.warning(
"observe ingest failed status=%s body=%s",
resp.status_code,
resp.text[:200],
)
async def _run_loop(client: httpx.AsyncClient) -> None:
try:
while True:
batch = await _collect_batch()
dropped = take_dropped()
if dropped:
logger.warning("observe dropped %d events (queue full)", dropped)
if not batch:
continue
try:
await _post_batch(client, batch)
except Exception: # noqa: BLE001 - best-effort 遥测,失败丢批不重试、不退出
logger.warning(
"observe post batch failed, dropped %d events",
len(batch),
exc_info=True,
)
except asyncio.CancelledError:
logger.info("observe worker stopped")
raise
def start_observe_worker() -> asyncio.Task | None:
"""启动上报 worker。未配置观测 → 返回 None(no-op)。"""
global _client
if not settings.observe_configured:
return None
_client = httpx.AsyncClient(
base_url=settings.OBSERVE_ENDPOINT,
auth=(settings.OBSERVE_USER, settings.OBSERVE_PASSWORD),
timeout=settings.OBSERVE_TIMEOUT_SEC,
)
logger.info(
"observe worker started endpoint=%s org=%s stream=%s",
settings.OBSERVE_ENDPOINT,
settings.OBSERVE_ORG,
settings.OBSERVE_STREAM,
)
return asyncio.create_task(_run_loop(_client), name="observe-worker")
async def stop_observe_worker(task: asyncio.Task | None) -> None:
"""收尾:cancel worker → best-effort 发最后一批 → 关 client。"""
global _client
if task is None:
return
task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await task
if _client is not None:
# worker 已停,安全 drain 剩余并 best-effort 发最后一批(短超时,不卡关停)。
try:
queue = get_queue()
final: list[dict] = []
while len(final) < settings.OBSERVE_BATCH_MAX:
try:
final.append(queue.get_nowait())
except asyncio.QueueEmpty:
break
if final:
await asyncio.wait_for(
_post_batch(_client, final), timeout=settings.OBSERVE_TIMEOUT_SEC
)
except Exception: # noqa: BLE001 - 关停期尽力而为,失败忽略
pass
await _client.aclose()
_client = None
```
- [ ] **Step 4: 运行,确认通过**
Run: `pytest tests/test_observe.py -q -k "collect_batch or post_batch or start_observe"`
Expected: PASS
- [ ] **Step 5: 提交**
```bash
git add app/core/observe_worker.py tests/test_observe.py
git commit -m "feat(observe): 加后台批量上报 worker(失败丢批不重试)"
```
---
## Task 5: 接线到 `app/main.py`(挂中间件 + lifespan 启停)
**Files:**
- Modify: `app/main.py`import 区、lifespan、CORS 之后)
- Test: `tests/test_observe.py`(追加)
- [ ] **Step 1: 写失败测试**
`tests/test_observe.py` 追加:
```python
def test_app_has_metrics_middleware():
from app.main import app
names = [m.cls.__name__ for m in app.user_middleware]
assert "RequestMetricsMiddleware" in names
```
- [ ] **Step 2: 运行,确认失败**
Run: `pytest tests/test_observe.py::test_app_has_metrics_middleware -q`
Expected: FAIL —— 断言失败(中间件尚未挂载)
- [ ] **Step 3: 实现接线**
3a. 在 `app/main.py` import 区(`withdraw_reconcile_worker` import 块之后)加:
```python
from app.core.observe import RequestMetricsMiddleware
from app.core.observe_worker import (
start_observe_worker,
stop_observe_worker,
)
```
3b. lifespan 里加启停(现有 `daily_exchange_task = start_daily_exchange_worker()` 之后、`try:` 之前加一行;`finally` 里在 `stop_daily_exchange_worker` 之后加一行):
```python
daily_exchange_task = start_daily_exchange_worker()
observe_task = start_observe_worker()
try:
yield
finally:
await stop_heartbeat_monitor(heartbeat_task)
await stop_withdraw_reconcile_worker(reconcile_task)
await stop_daily_exchange_worker(daily_exchange_task)
await stop_observe_worker(observe_task)
await aclose_pricebot_client()
logger.info("shutting down")
```
3c. 挂中间件——在 CORS 的 `if settings.cors_origins_list:` 整块之后加(使其成为最外层,测到含 CORS 的完整耗时):
```python
# 接口指标埋点(最外层:测含 CORS 的完整耗时)。未配置观测时中间件自 no-op。
app.add_middleware(RequestMetricsMiddleware)
```
- [ ] **Step 4: 运行,确认通过**
Run: `pytest tests/test_observe.py::test_app_has_metrics_middleware -q`
Expected: PASS
- [ ] **Step 5: 跑整套 observe 测试 + 全量回归,确认无破坏**
Run: `pytest tests/test_observe.py -q && pytest -q`
Expected: 全 PASS(现有用例不受影响:conftest 未设 `OBSERVE_*` → 观测关 → worker no-op、中间件透传)
- [ ] **Step 6: 提交**
```bash
git add app/main.py tests/test_observe.py
git commit -m "feat(observe): main.py 挂中间件 + lifespan 启停上报 worker"
```
---
## Task 6: OpenObserve 本地部署(compose + README + .env.example
**Files:**
- Create: `deploy/openobserve/docker-compose.yml`
- Create: `deploy/openobserve/README.md`
- Modify: `.env.example`(追加 `OBSERVE_*` 段)
- [ ] **Step 1: 写 docker-compose**
新建 `deploy/openobserve/docker-compose.yml`
```yaml
# 本地开发用 OpenObserve(单容器 = local 模式)。用于接收 app-server 的接口指标。
# 启动: cd deploy/openobserve && docker compose up -d
# Web UI: http://localhost:5080 (账号见下方 env)
services:
openobserve:
image: public.ecr.aws/zinclabs/openobserve:latest
container_name: openobserve
ports:
- "5080:5080"
environment:
ZO_ROOT_USER_EMAIL: "admin@shaguabijia.local"
ZO_ROOT_USER_PASSWORD: "Complexpass#123"
ZO_DATA_DIR: "/data"
volumes:
- ./data:/data
restart: unless-stopped
```
- [ ] **Step 2: 写 README**
新建 `deploy/openobserve/README.md`
````markdown
# OpenObserve 本地部署(接口 QPS / 耗时可观测)
app-server 通过中间件采集每个接口的 QPS + 耗时 + 错误率,批量上报到这里。
## 启动
```bash
cd deploy/openobserve
docker compose up -d
```
- Web UIhttp://localhost:5080
- 登录:`admin@shaguabijia.local` / `Complexpass#123`(见 `docker-compose.yml`
- 数据落 `deploy/openobserve/data/`(已挂卷持久化;`data/` 建议 gitignore
## 让 app-server 上报
在项目根的 `.env` 打开观测(`OBSERVE_*`,账号密码与 compose 里 root 一致):
```dotenv
OBSERVE_ENABLED=true
OBSERVE_ENDPOINT=http://localhost:5080
OBSERVE_ORG=default
OBSERVE_STREAM=app_requests
OBSERVE_USER=admin@shaguabijia.local
OBSERVE_PASSWORD=Complexpass#123
```
重启 app-server,随便打几个接口。stream `app_requests` **首次上报自动创建**
在 UI 的 Logs → 选 `app_requests` 就能看到逐条请求事件。
## 查询(Logs 页 SQL,或建 Dashboard 面板)
各接口 QPS(1 分钟分桶,面板里再除 60 得每秒):
```sql
SELECT route, histogram(_timestamp, '1 minute') AS ts, count(*) AS cnt
FROM app_requests GROUP BY route, ts ORDER BY ts
```
各接口 P95 耗时(毫秒):
```sql
SELECT route, approx_percentile_cont(duration_ms, 0.95) AS p95_ms
FROM app_requests GROUP BY route ORDER BY p95_ms DESC
```
各接口错误率(5xx 占比):
```sql
SELECT route,
count(*) FILTER (WHERE status >= 500) * 100.0 / count(*) AS err_pct
FROM app_requests GROUP BY route ORDER BY err_pct DESC
```
## 停止 / 清数据
```bash
docker compose down # 停止(保留数据)
docker compose down -v && rm -rf data # 停止并清空数据
```
> 生产部署(持久化规格、独立 ingest 账号、鉴权收紧)见 spec 第 9 节,本期不做。
````
- [ ] **Step 3: 追加 `.env.example`**
`.env.example` 末尾追加:
```dotenv
# ===== 可观测(OpenObserve 接口指标)=====
# 采集每个接口 QPS + 耗时 + 错误率,批量直采到 OpenObserve(本地 Docker,见 deploy/openobserve/)。
# 默认关;开启需 ENABLED=true 且填 USER/PASSWORD(与 docker-compose 里 root 账号一致)。
OBSERVE_ENABLED=false
OBSERVE_ENDPOINT=http://localhost:5080
OBSERVE_ORG=default
OBSERVE_STREAM=app_requests
OBSERVE_USER=admin@shaguabijia.local
OBSERVE_PASSWORD=Complexpass#123
# 进阶(一般不用改):攒批间隔秒 / 单批最大条数 / 有界队列上限(满则丢) / 上报超时秒
OBSERVE_FLUSH_INTERVAL_SEC=5
OBSERVE_BATCH_MAX=200
OBSERVE_QUEUE_MAX=10000
OBSERVE_TIMEOUT_SEC=5
```
- [ ] **Step 4: 校验 compose 语法(不需真拉镜像)**
Run: `docker compose -f deploy/openobserve/docker-compose.yml config`
Expected: 打印规整后的配置、无报错(若本机无 docker,可跳过,标注为手动验证项)
- [ ] **Step 5: 提交**
```bash
git add deploy/openobserve/docker-compose.yml deploy/openobserve/README.md .env.example
git commit -m "feat(observe): 加 OpenObserve 本地 compose + README + .env.example"
```
---
## Task 7: 端到端手动验证 + 全量 lint/test 收尾
**Files:** 无(验证 + 收尾)
- [ ] **Step 1: 起 OpenObserve**
Run: `cd deploy/openobserve && docker compose up -d`
Expected: 容器起来,浏览器打开 http://localhost:5080 能登录
- [ ] **Step 2: 本地开观测起 app-server**
在根 `.env``OBSERVE_ENABLED=true` + `OBSERVE_USER/PASSWORD`(同 compose),然后:
Run: `./run.sh`Windows 用 `python -m uvicorn app.main:app --port 8770 --reload --reload-dir app`
Expected: 启动日志出现 `observe worker started endpoint=http://localhost:5080 ...`
- [ ] **Step 3: 打几个接口产生数据**
Run: `curl http://localhost:8770/health && curl http://localhost:8770/things-does-not-exist -i`(或正常业务接口若干)
Expected: 稍等 ≤5sflush 间隔),OpenObserve UI 的 Logs → `app_requests` 出现事件;`/health` 不应出现;不存在的路径 route 为 `__unmatched__`
- [ ] **Step 4: 验证三条查询**
在 OpenObserve UI 分别粘贴 README 里的 QPS / P95 / 错误率 SQL,确认能出数。
- [ ] **Step 5: lint(仅本改动涉及文件)+ 全量测试**
> 说明:仓库基线有 ~558 个既有 ruff 错误、且未强制 ruff 通过。不要去清历史欠债(范围蔓延)。只要求**本次新增/改动的文件**零 ruff 错误。
Run: `ruff check app/core/observe.py app/core/observe_worker.py tests/test_observe.py && python -m pytest -q`
Expected: 上述三个新文件 ruff 无错;测试里 `tests/test_observe.py` 全 PASS,且**全量失败数不超过基线的 4 个**(test_compare_proxy ×2 / test_coupon_proxy ×1 / test_invite ×1,均与本功能无关)。
额外确认我对既有文件的改动没有引入**新的** ruff 错误:`ruff check app/core/config.py app/main.py`(数量应与基线一致,不因本改动增加)。
- [ ] **Step 6: 关观测复跑一次,确认降级**
`.env``OBSERVE_ENABLED` 改回 `false``ruff check .` 不涉及,直接 `pytest -q`
Expected: 全 PASS(验证 observe 关闭时零副作用)
- [ ] **Step 7: 收尾提交(如有 .env 之外的改动)**
```bash
git add -A
git commit -m "chore(observe): 端到端验证与收尾" --allow-empty
```
> `.env` 不入 git(已 gitignore);本任务只验证,不提交 `.env`
---
## Self-Review(写完计划后自查)
- **Spec 覆盖**Docker 部署→Task 6/7;事件 schema→Task 3`record_event` 事件字段);中间件→Task 3worker→Task 4;配置→Task 1main 接线→Task 5;查询/仪表盘→Task 6 README;测试→Task 1-5;决策(a)队列满丢→Task 2(b)失败不重试→Task 4(c)跳过 /health→Task 3。全覆盖。
- **占位符**:无 TBD/TODO;每个代码步骤含完整代码。
- **类型/命名一致**`get_queue` / `record_event` / `take_dropped` / `RequestMetricsMiddleware` / `start_observe_worker` / `stop_observe_worker` / `observe_configured` / 事件字段名,跨 Task 1-5 与文件结构表一致。
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,236 @@
# 接口 QPS + 耗时可观测(OpenObserve)设计
- **日期**2026-07-06
- **状态**:已评审通过,待写实现计划
- **范围**:仅 app-server8770);admin8771)暂不接入
- **方案**:A —— 轻量自研 ASGI 中间件 + 后台 worker 批量直采到 OpenObserve
## 1. 背景与目标
app-server 目前除 CORS 外无任何中间件,也无接口级可观测。需要按**每个接口**采集:
- **QPS**(每秒请求数,可按接口/时间分桶)
- **耗时**P50/P95/P99 等分位)
顺带低成本拿到**错误率**`status >= 500` 占比)。落地目标是:本地 Docker 跑一个 OpenObserve 实例接收数据,服务侧加埋点上报,在 OpenObserve 仪表盘上看各接口 QPS + 耗时。
### 非目标(YAGNI
- 不做分布式 trace / span 关联(只要接口聚合指标)。
- 不引入 OpenTelemetry / Prometheus 客户端等重依赖。
- 不采集请求体 / query / 用户身份等,任何 PII 都不进上报。
- admin(8771)本期不接(中间件写成可复用,未来一行挂载即可)。
- 上报失败不做持久化重试 / 落盘补偿(best-effort)。
## 2. 方案选型
对比过三条路(详见评审记录):
- **A 轻量自研中间件 + JSON 直采**(选中):零新依赖(`httpx` 已在依赖里),完全贴合本仓库「后台 worker + JSON 事件 + `*_configured` 优雅降级」的既有习惯,恰好满足「每接口 QPS + 耗时 + 错误率」并保留原始事件下钻能力。
- B OpenTelemetry 自动埋点 + OTLP:行业标准、顺带 trace,但多 5–6 个依赖、概念多、数据量/成本高于需求,与精简代码库风格相悖。
- C Prometheus 进程内聚合 + remote_write/抓取:数据量最小,但 remote_write 编码复杂或需额外抓取进程,丢失单请求下钻,最不贴合 OpenObserve 的 log-first 强项。
**结论:A。**
## 3. 架构与数据流
```
每个 HTTP 请求
→ RequestMetricsMiddleware(最外层:测总耗时 / 抓路由模板 + 状态码)
→ record_event() 非阻塞入队(有界队列,满则丢最旧,绝不阻塞、绝不 OOM)
→ observe_worker(后台 asyncio.Task,随 lifespan 启停)批量 drain
→ httpx POST {ENDPOINT}/api/{ORG}/{STREAM}/_json → OpenObserve
→ 仪表盘 SQL 聚合出 QPS / 分位耗时 / 错误率
```
**核心不变量**
1. 请求路径上只做「测时 + 构建一个小 dict + `put_nowait`」,**无任何网络/磁盘 I/O**。
2. 所有上报 I/O 在后台 worker;worker 捕获全部异常,绝不让埋点影响请求。
3. 未配置观测(`observe_configured=False`)→ 中间件透传、worker 不启动,整套 no-op。
4. OpenObserve 不可用 → 队列填满后丢弃事件 + 限流告警,业务零影响。
## 4. 组件设计
### 4.1 OpenObserve 本地部署 —— `deploy/openobserve/docker-compose.yml`(新增)
```yaml
services:
openobserve:
image: public.ecr.aws/zinclabs/openobserve:latest
container_name: openobserve
ports: ["5080:5080"]
environment:
ZO_ROOT_USER_EMAIL: "admin@shaguabijia.local"
ZO_ROOT_USER_PASSWORD: "Complexpass#123"
ZO_DATA_DIR: "/data"
volumes: ["./data:/data"]
restart: unless-stopped
```
- `docker compose up -d` 启动;Web UI `http://localhost:5080`,用上面邮箱/密码登录。
- 单容器 = local 模式,数据落 `./data`(已挂卷持久化)。
- **stream 首次上报自动创建**,无需预建 `app_requests`
- 上报鉴权:HTTP Basic auth`email:password`),本地直接用 root 账号;生产应另建仅具 ingest 权限的用户/服务账号(本期不涉及)。
### 4.2 事件 schema(一请求一行 JSON
```json
{
"_timestamp": 1720000000000000, // 微秒(µs)整数,请求完成时刻。OpenObserve 默认时间列 _timestamp 以微秒计
"service": "app-server", // 取 LOG_SERVICE_NAME / 固定值
"env": "dev", // settings.APP_ENV
"method": "POST",
"route": "/api/v1/coupon/step", // 路由模板(非实际 path)
"status": 200,
"duration_ms": 42.7 // float 毫秒
}
```
- **只存路由模板**(如 `/c/{code}``/media` 静态归一),避免 path 参数把维度打爆。
- 未匹配路由(404 / 扫描器)归一到常量 `__unmatched__`
- 只采 method / route / status / duration —— 无 body、无 query、无 PII。
### 4.3 埋点中间件 —— `app/core/observe.py`(新增)
**纯 ASGI 中间件**(比 `BaseHTTPMiddleware` 开销低;能可靠读到路由与最终状态码;scope 按引用透传,内层 router 的 `scope["route"]` 外层可见)。
职责:
1. 非 `http` 请求、或 `not settings.observe_configured` → 直接透传,不测。
2. `perf_counter()` 记起点;包一层 `send``http.response.start``status`(默认兜底 500,覆盖下游抛异常未产出 response 的情况)。
3. `finally` 里算 `duration_ms`,从 `scope` 取路由模板(见下),构建事件,调 `record_event()`
4. 跳过路径集合 `_SKIP_PATHS = {"/health"}`(纯噪音)。
**路由模板解析(跨 Starlette 版本稳健)**
```python
route = scope.get("route")
template = getattr(route, "path", None)
if template is None: # 未匹配 / 老版本未写 scope["route"]
template = "__unmatched__"
```
(若实测某 Starlette 版本不写 `scope["route"]`,回退用 `request.app.router.routes` 逐个 `route.matches(scope)==Match.FULL` 找模板;实现时以实际版本为准,优先 `scope["route"]`。)
**入队(`record_event`**:模块级 `asyncio.Queue(maxsize=OBSERVE_QUEUE_MAX)`。用 `put_nowait``QueueFull` 则丢弃并累加一个 `_dropped` 计数(每累计 N 条限流打一条 WARNING)。**永不 `await put()`、永不阻塞请求**。
> 决策(a):队列满 → **丢弃**(不阻塞请求)。
### 4.4 上报 worker —— `app/core/observe_worker.py`(新增)
对齐现有 `heartbeat_monitor_worker.py` / `daily_exchange_worker.py` / `withdraw_reconcile_worker.py``start_*` / `stop_*` 形态。
- `start_observe_worker() -> asyncio.Task | None`
- `not settings.observe_configured` → 返回 `None`no-op)。
- 否则建专用 `httpx.AsyncClient``base_url=ENDPOINT``auth=(USER, PASSWORD)``timeout=OBSERVE_TIMEOUT_SEC`),起 `_run_loop` task。
- `_run_loop()`:循环
1. `_collect_batch()``await asyncio.wait_for(queue.get(), timeout=FLUSH_INTERVAL)` 拿到首条(超时且空 → 返回空,continue);再 `get_nowait()` 连抽到 `BATCH_MAX` 条或抽空。
2. `POST /api/{ORG}/{STREAM}/_json`body 为事件数组。
3. **catch 所有异常**:失败限流打 WARNING,**直接丢弃该批,不重试**。
- `stop_observe_worker(task)`best-effort 收尾 flush(短超时)→ `task.cancel()``await`(吞 `CancelledError`)→ 关 client。
> 决策(b):上报失败 → **直接丢弃,不重试**best-effort 遥测)。
### 4.5 配置 —— `app/core/config.py`(改)
新增一段 `# ===== 可观测(OpenObserve 接口指标)=====`,默认全关(prod 安全):
| 配置 | 默认 | 说明 |
|---|---|---|
| `OBSERVE_ENABLED` | `False` | 总开关;默认关,opt-in |
| `OBSERVE_ENDPOINT` | `http://localhost:5080` | OpenObserve base URL |
| `OBSERVE_ORG` | `default` | 组织名 |
| `OBSERVE_STREAM` | `app_requests` | stream 名 |
| `OBSERVE_USER` | `""` | Basic auth 邮箱 |
| `OBSERVE_PASSWORD` | `""` | Basic auth 密码/token |
| `OBSERVE_FLUSH_INTERVAL_SEC` | `5.0` | worker 最长攒批间隔 |
| `OBSERVE_BATCH_MAX` | `200` | 单批最大事件数 |
| `OBSERVE_QUEUE_MAX` | `10000` | 有界队列上限,满则丢 |
| `OBSERVE_TIMEOUT_SEC` | `5.0` | 上报 HTTP 超时 |
```python
@property
def observe_configured(self) -> bool:
return bool(self.OBSERVE_ENABLED and self.OBSERVE_ENDPOINT
and self.OBSERVE_USER and self.OBSERVE_PASSWORD)
```
`.env.example` 同步补一段带注释的 `OBSERVE_*`(沿用该文件重注释风格),`OBSERVE_ENABLED=false`
### 4.6 接线 —— `app/main.py`(改)
- import `RequestMetricsMiddleware``start_observe_worker` / `stop_observe_worker`
- `app.add_middleware(RequestMetricsMiddleware)`:放在 CORS `add_middleware` **之后** → 成为最外层,测到含 CORS 的完整耗时。无条件挂载(内部自 no-op)。
- `lifespan`:启动 `observe_task = start_observe_worker()``finally``await stop_observe_worker(observe_task)`,与现有 worker 并列。
### 4.7 OpenObserve 查询 / 仪表盘 —— `deploy/openobserve/README.md`(新增)
含:compose 启停、登录、stream 自动创建说明、`.env` 接线,以及可直接粘的示例 SQL
- **各接口 QPS**1 分钟分桶):
```sql
SELECT route, histogram(_timestamp, '1 minute') AS ts, count(*) AS cnt
FROM app_requests GROUP BY route, ts ORDER BY ts
```
(面板按 `cnt/60` 展示每秒;或用 OpenObserve 图表的 rate 能力。)
- **各接口 P95 耗时**
```sql
SELECT route, approx_percentile_cont(duration_ms, 0.95) AS p95_ms
FROM app_requests GROUP BY route ORDER BY p95_ms DESC
```
- **各接口错误率**
```sql
SELECT route,
count(*) FILTER (WHERE status >= 500) * 100.0 / count(*) AS err_pct
FROM app_requests GROUP BY route ORDER BY err_pct DESC
```
## 5. 关键设计决策汇总
- **(a) 队列满 → 丢弃**(不阻塞请求):遥测让路于业务可用性。
- **(b) 上报失败 → 不重试**best-effort;避免 poison batch 堆积与队列无限增长。
- **(c) 跳过 `/health`**:健康检查是纯噪音,硬编码在 `_SKIP_PATHS`
- **只存路由模板 + `__unmatched__`**:防维度爆炸。
- **默认 OFF、opt-in**:prod 安全默认;开启后仍全异步 + 有界。
- **纯 ASGI 中间件 + `perf_counter`**:请求路径开销微秒级,无 I/O。
## 6. 安全 / 性能保证
- 请求路径新增开销 ≈ 一次 `perf_counter` 差 + 一个小 dict + 一次 `put_nowait`(微秒级),无锁竞争的显著热点。
- 失败隔离:入队丢弃 + worker 全异常捕获;OpenObserve 宕机不影响任何请求。
- 有界内存:队列 `maxsize` 封顶,最坏丢事件不涨内存。
- 无 PII:仅 method / route / status / duration。
## 7. 测试策略 —— `tests/test_observe.py`(新增)
沿用仓库约定(`TestClient` + `monkeypatch`,绝不打真网络;`conftest` 在 import 前设 env):
1. 埋点入队字段正确:模板路由、`status``duration_ms > 0`
2. 参数化路由 → 取到**模板**而非实际 path。
3. 未匹配路径(404)→ `route == "__unmatched__"`
4. `OBSERVE_ENABLED=false` → 零入队、零 HTTP(现有测试不受影响)。
5. 队列满 → `record_event` 不抛异常(走丢弃分支)。
6. worker 批量 POST 的 URL / payload 正确(monkeypatch httpx client / `_post`,不打网络)。
7. `/health` 被跳过 → 不入队。
> `settings``lru_cache` 单例;需要开启观测的用例通过 monkeypatch `settings` 属性或直接调 `record_event` / 中间件并 patch `observe_configured` 实现,避免全局 env 改动波及他用例。
## 8. 文件清单
| 文件 | 动作 |
|---|---|
| `deploy/openobserve/docker-compose.yml` | 新增(OpenObserve 容器)|
| `deploy/openobserve/README.md` | 新增(部署步骤 + 查询/仪表盘)|
| `app/core/observe.py` | 新增(中间件 + 有界队列 + `record_event` + 路由解析)|
| `app/core/observe_worker.py` | 新增(后台批量上报 worker)|
| `app/core/config.py` | 改(`OBSERVE_*` + `observe_configured`|
| `app/main.py` | 改(挂中间件 + lifespan 启停 worker|
| `.env.example` | 改(新增 `OBSERVE_*` 注释段)|
| `tests/test_observe.py` | 新增 |
## 9. 未来工作(本期不做)
- admin(8771)接入同一套中间件(`service` 字段区分)。
- 生产部署 OpenObserve(持久化、独立 ingest 账号、资源规格、鉴权收紧)。
- 上报字段扩展(如按 user/设备维度、上游 pricebot 透传耗时拆分)。
@@ -0,0 +1,296 @@
# 15 天不活跃自动清零(金币 + 现金)设计
- **日期**2026-07-16
- **状态**Draft — 待评审
- **所属**app-server`app/`),含一处 admin 侧重构 + 一项 Android 端埋点依赖
- **一句话**:连续 15 天不活跃的用户,自动清零其金币与现金;清零前按可配置节奏预警;全过程留审计以备纠纷排查。
---
## 1. 背景与目标
运营需要对**长期不活跃**用户的钱包余额做清理。两条硬性要求:
1. **可审计**:记录清零原因与**清零前的三桶余额**,便于后续排查与处理客户纠纷。
2. **临清预警**:在临近清零前推送信息告知用户"因账号不活跃,账户里的 xx 金币和 xx 现金将被清零"。
### 非目标(本期不做)
- 不做真实推送通道(极光 JPush / 短信)的对接 —— 仅做**可插拔通知器 + 日志占位**,接口预留、后续无缝替换。
- 不改动提现(`WithdrawOrder`)流程。
- 不新增 `User.last_active_at` 列、不改鉴权热路径。
---
## 2. 需求
| # | 需求 | 落地 |
|---|---|---|
| R1 | 连续 15 天不活跃 → 清零金币 + 现金 | 每日 worker 扫描 + 逐用户事务清零(§6) |
| R2 | 记录清零原因 + 清零前余额 | `inactivity_reset_log` 审计表 + 3 条钱包流水(§5、§7) |
| R3 | 临清前预警"xx 金币 xx 现金将清零" | 阶段 A 预警 + `inactivity_notification_log`(§6、§7 |
| R4 | 活跃口径与"用户管理"一致 | 抽共享模块 `activity.py`admin 与 worker 共用(§4、§12 |
| R5 | 预警时机完全可配置 | `INACTIVITY_*` 配置项(§8 |
---
## 3. 决策记录(来自评审问答)
| 决策点 | 结论 | 理由 |
|---|---|---|
| **活跃口径** | 与"用户管理"一致:`max(首页可见 show/home, 比价, 领券)`**不含 last_login_at**;无任何信号时以 `created_at` 为非空基线 | 比价可从**浮窗**触发、不进首页;`last_login_at` 只在登录/换绑动作更新(re-login 也算),代表不了"在用 App",故彻底排除 |
| **"进首页"信号落地** | **方案 A:前端上报 `home_view` 埋点**(复用 `/analytics/events`),非新接口 | 三个活跃信号统一为同类埋点事件;零新接口零新列;与 admin 口径天然一致。B(鉴权接口 + 列)"更权威"的优势是假的——比价/领券仍是端上报事件,最弱环决定整体可信度 |
| **清零范围** | **金币 + 折算现金**(**邀请现金不清**——产品红线,仅快照入审计) | 对应"账户里的金币和现金";邀请奖励金与金币现金物理隔离、不可累加,见 `wallet.CoinAccount` 注释 |
| **预警推送** | **可插拔通知器 + 日志占位**v1),后续接 JPush/短信 | 现状无真实推送能力;先把清零主流程 + 审计做扎实,不阻塞 |
| **预警时机** | **完全可配置**(提前天数列表 + 次数 + 执行点 + 通道) | R5 |
| **触发方式** | **进程内每日 worker**,仿 `daily_exchange_worker` | 与项目最新模式一致,无需外部 cron |
| **admin 共享口径** | 共享模块 + **重构 admin 改用它** | 单一真源,永不漂移(R4 |
### 已知取舍(可接受)
- analytics 的 `user_id` 是**端上报、未鉴权**(可伪造)。但伪造只能"保自己活跃、避免被清",无收益,且正是本功能要防的行为,风险良性。活跃时间的非空基线由服务端权威的 `User.created_at` 提供(见 §4),不再依赖 `last_login_at`。与"用户管理"口径一致。
---
## 4. 活跃口径与共享模块 `app/repositories/activity.py`(新建)
活跃口径的**唯一真源**。app 侧模块,admin 可 import`app.main` 不 import `app.admin`,反向允许)。
### 口径
```
last_active = max(
User.created_at, # 注册基线(恒非空;re-login 不推进,只有真实使用才推进)
max AnalyticsEvent.created_at WHERE event IN ACTIVE_EVENTS,
max CouponPromptEngagement.created_at WHERE engage_type == "claim_started",
)
不活跃判定:按北京自然日、0 点对齐(非从末次活跃时刻滚动 15×24h)
last_active_date = 北京(last_active).date() # 末次活跃的北京日,记为「第 1 日」
清零边界 = 北京 00:00 of (last_active_date + RESET_DAYS 天) =「第 (RESET_DAYS+1) 日 0 点」 # 15 → 第16日0点
应清零 ⟺ (cn_today() last_active_date).days ≥ RESET_DAYS
⟺ last_active < cutoff cutoff = 北京 00:00 of (cn_today() (RESET_DAYS1)) # 供 SQL 比较
inactive_days = (cn_today() last_active_date).days # 清零当日恰 = RESET_DAYS
例:末次活跃 1/1 → 1/16 00:00(第16日0点)清零,当日 inactive_days=151/15 及之前不清
```
### 模块内容
- 常量:
- **首页可见活跃信号已定名:`event=show` + `page=home`**(前端确认,原占位 `home_view`;下文出现的 `home_view` 均指此信号)。活跃行为过滤见 `activity.active_event_condition()`:首页可见 比价 `real_compare_start` 领券 `real_coupon_start``ACTIVE_EVENTS` 仅含后两个纯 event 名(首页可见是 event+page 组合、单列)。
- `ACTIVE_ENGAGE_TYPE = "claim_started"`
- `last_active_subqueries(db)` —— 复刻现 admin `queries._last_active_parts()`:两个按 `user_id``GROUP BY max(created_at)` 聚合子查询。
- `last_active_expr(base_col, ev_sub, eng_sub, dialect)` —— 生成 `greatest`/`max`PG `func.greatest`SQLite `func.max`);子聚合缺失时 `coalesce(子聚合, User.created_at)` 兜底(注册基线恒非空,**替代原 last_login_at**)。
- `_norm_utc()` —— 沿用现 admin 的 naive→UTC 归一(SQLite naive / PG aware 混算保护)。
- `reset_cutoff(reset_days)` / `warn_cutoff(reset_days, k)` —— 生成**北京 0 点对齐**的边界 datetime(见口径):`reset_cutoff = 北京 00:00 of (cn_today() (reset_days1))`,供下面查询按 `last_active < cutoff` 比较。
- `select_inactive_users(db, *, cutoff, with_balance=True)` —— **worker 专用**join `CoinAccount`,筛 `last_active < cutoff`(cutoff = 北京 0 点对齐边界,见口径)且(`coin_balance>0 OR cash_balance_cents>0`;**邀请现金不清、不计入候选**),返回 `(user, account, last_active, inactive_days)`
- `select_warn_targets(db, *, reset_days, warn_days_before)` —— **worker 专用**:返回 `(user, account, last_active, inactive_days, stage)` 元组——各"提前天数"窗口内、有余额、本 streak 未推过档 `stage` 的用户(去重结合 `notification_log`,逻辑见 §9)。
> **参考现状**:现口径散落在 `app/admin/repositories/queries.py:38,91-124,199-204``_ACTIVE_EVENTS`/`_last_active_parts`/`greatest`)与 `app/admin/repositories/stats.py:51-52,138-146``COMPARE_START_EVENT`/`COUPON_START_EVENT`/活跃用户集)。这些改为从 `activity.py` 导入(§12)。
---
## 5. 数据模型(2 张新表,不动 `User`
两表均登记进 `app/models/__init__.py`;一个 Alembic 迁移建两表(`render_as_batch`SQLite 兼容)。
### ① `inactivity_reset_log` —— 清零审计(R2
仿 `app/models/phone_rebind_log.py` 的简单审计表风格。
| 字段 | 类型 | 说明 |
|---|---|---|
| `id` | int PK autoincrement | |
| `user_id` | int, index, not null | |
| `coin_balance_before` | int, not null | 清零前金币 |
| `cash_balance_cents_before` | int, not null | 清零前折算现金(分) |
| `invite_cash_balance_cents_before` | int, not null | 清零前邀请现金(分) |
| `last_active_at` | DateTime(tz), nullable | 判定时的最近活跃时间 |
| `inactive_days` | int, not null | 判定时不活跃天数 |
| `reason` | String(32), not null | 如 `"inactive_15d"` |
| `reset_at` | DateTime(tz), server_default now(), index, not null | 清零时刻 |
### ② `inactivity_notification_log` —— 预警记录 + 去重 + 占位 outboxR3
| 字段 | 类型 | 说明 |
|---|---|---|
| `id` | int PK autoincrement | |
| `user_id` | int, index, not null | |
| `stage` | int, not null | 提前天数档(如 7 / 2 |
| `inactive_days` | int, not null | 推送时不活跃天数 |
| `coin_balance` | int, not null | 推送快照:告知用户的金币数 |
| `cash_balance_cents` | int, not null | 推送快照:折算现金 |
| `invite_cash_balance_cents` | int, not null | 推送快照:邀请现金 |
| `channel` | String(16), not null | `"log"` / `"jpush"` / `"sms"` |
| `status` | String(16), not null | `"placeholder"` / `"sent"` / `"failed"` |
| `created_at` | DateTime(tz), server_default now(), index, not null | 去重锚点(见 §9 |
> 备注:不新增 `User.last_active_at` 列,不改 `get_current_user`。活跃时间由 §4 口径**实时计算**。
---
## 6. 清零 worker `app/core/inactivity_reset_worker.py`(新建)
**完全仿 [`app/core/daily_exchange_worker.py`](../../../app/core/daily_exchange_worker.py)**App 启动自带 asyncio 任务,文件锁(`data/inactivity_reset.lock`)防同机多进程并发。**worker 常驻**;`INACTIVITY_RESET_ENABLED` 只决定是否**真清**:false(默认)= 只记审计名单、不动钱(dry-run),true = 真清。
### 调度
- 每 `INACTIVITY_RESET_CHECK_INTERVAL_SEC` 秒醒一次;`last_run: date` 守卫**北京日**,保证每日只跑一轮。
- 仅当 `cn_today() != last_run` 且当前北京小时 `>= INACTIVITY_RESET_RUN_HOUR` 时执行(启动补跑同 daily_exchange 语义)。
- **清零资格边界 = 第 16 日 0 点(北京,见 §4),与 worker 执行点解耦**worker 于当日 `RUN_HOUR`(默认 3 点)跑,把已过边界者一并清;若要严格 0 点触发可置 `RUN_HOUR=0`,但注意与 `daily_auto_exchange` 的 0 点任务错峰。
- lifespan 里 `start_inactivity_reset_worker()` / `stop_...`(仿 `start_daily_exchange_worker``app/main.py` 的接线)。
### 一轮 `run_once(db)` 两阶段(同一次运行、各自逐用户独立 commit)
**阶段 A — 预警**
```
for user, acc, last_active, inactive_days, stage in activity.select_warn_targets(...):
notifier.send_inactivity_warning(user, balances=snapshot(acc), stage=stage, days_until_reset=RESET_DAYS-inactive_days)
db.add(InactivityNotificationLog(..., channel=notifier.channel, status=notifier.last_status))
db.commit() # 逐条独立
```
**阶段 B — 清零**`biz_type="inactivity_reset"`
```
for user, acc, last_active, inactive_days in activity.select_inactive_users(db, cutoff=activity.reset_cutoff(RESET_DAYS)): # 北京 00:00 of (今天−(RESET_DAYS1))
try:
acc = wallet.get_or_create_account(db, user.id, commit=False, lock=True) # 行锁
before = (acc.coin_balance, acc.cash_balance_cents, acc.invite_cash_balance_cents)
if acc.coin_balance == 0 and acc.cash_balance_cents == 0: continue # 邀请现金不清,不算可清余额
log = InactivityResetLog(user_id=user.id, coin_balance_before=before[0],
cash_balance_cents_before=before[1], invite_cash_balance_cents_before=before[2], # 邀请现金仅快照
last_active_at=last_active, inactive_days=inactive_days, reason=f"inactive_{RESET_DAYS}d")
db.add(log); db.flush() # 拿 log.id 作 ref_id 交叉链接
if acc.coin_balance: wallet.grant_coins(db, user.id, -acc.coin_balance, biz_type="inactivity_reset", ref_id=str(log.id), remark="15天不活跃清零")
if acc.cash_balance_cents: wallet.grant_cash(db, user.id, -acc.cash_balance_cents, biz_type="inactivity_reset", ref_id=str(log.id), remark="15天不活跃清零")
# 邀请现金(invite_cash_balance_cents)不清:产品红线、两本账物理隔离,仅快照记入审计。
db.commit()
except SQLAlchemyError:
db.rollback(); stats["failed"] += 1
```
- `grant_*` 负数出账、`balance_after=0`、写**两条**流水(金币 + 折算现金;**邀请现金不清**);`grant_coins` 负数**不**动 `total_coin_earned`(历史累计保留)。
- 逐用户独立 commit:一个失败不影响其余。返回 `stats = {warned, warn_skipped, warn_failed, scanned, cleared, failed}``logger.info`。**预警逐用户 try/except 隔离、且预警整段异常也绝不阻塞清零**(清零是不可逆资金操作,不能被通知故障拖住)。
---
## 7. 预警与可插拔通知器
`app/integrations/notifier.py` 定义协议(外部投递属 integrations 层):
```python
class InactivityNotifier(Protocol):
channel: str # "log" / "jpush" / "sms"
last_status: str # "placeholder" / "sent" / "failed"
def send_inactivity_warning(self, user, *, balances, stage, days_until_reset) -> None: ...
```
- **v1 `LogNotifier`**`channel="log"`):`logger.warning("[inactivity-warn] user=%s coin=%s cash=%s invite=%s T-%s", ...)``last_status="placeholder"`。参照 `heartbeat_monitor_worker` 先例("本期先不接推送,用终端打印代替")。
- 未来 `JPushNotifier` / `SmsNotifier`:实现同协议即可替换,worker 不改。
- 选择:`INACTIVITY_NOTIFY_CHANNEL` → 工厂返回对应实现(未配到真实实现时回退 `LogNotifier`)。
- 预警文案数据来自快照 `balances`,满足 R3"告知 xx 金币 xx 现金"。
---
## 8. 配置项(`app/core/config.py`
```
INACTIVITY_RESET_ENABLED = False # false(默认)=只记审计名单(dry-run,不动钱);true=真清
INACTIVITY_RESET_DAYS = 15 # 不活跃阈值(天)
INACTIVITY_WARN_DAYS_BEFORE = "7,2" # 清零前几天各推一次;空串=不推。逗号分隔,降序解析
INACTIVITY_RESET_RUN_HOUR = 3 # 北京时间每日执行点(0-23)
INACTIVITY_NOTIFY_CHANNEL = "log" # log(占位) / jpush / sms
INACTIVITY_RESET_CHECK_INTERVAL_SEC = 1800 # worker 唤醒间隔(可复用现有间隔常量)
```
- 清零范围(三桶)固定为常量,不做配置。
- `INACTIVITY_WARN_DAYS_BEFORE` 语义(`inactive_days` 为北京自然日,见 §4):档位 `k` ⟹ 当 `inactive_days >= RESET_DAYS-k``< RESET_DAYS` 且本 streak 未推过档 `k` 时预警,即在北京日 `last_active_date + (RESET_DAYSk)` 触发(漏跑某天时补发最紧急未推档,§9)。
- `INACTIVITY_RESET_RUN_HOUR` 只决定 worker 每日执行点,**不改变**"第 16 日 0 点"这一资格边界(§4/§6)。
---
## 9. 幂等与重新活跃
- **重新活跃自动退出**`inactive_days` 由 §4 口径**实时算**。用户一有 `home_view`/比价/领券(**登录本身不算**),`last_active` 前移,自动移出预警与清零队列。**无需**显式"重置标记"。
- **预警去重**`inactivity_notification_log` 中存在 `stage==k 且 created_at > last_active` 的行 ⟹ 本 streak 已推过档 `k`,不重推。用户回归后 `last_active` 前移,旧预警行自然"失效",开启新 streak。
- **清零幂等**:阶段 B 只处理三桶非全 0 者;清完 = 0,次日不再匹配。worker 重启 / 多次唤醒 / 补跑均安全,不产生重复清零或重复流水。
- **稳健补发**:worker 漏跑数天后,某用户可能同时满足多档;只补发**最紧急的未推档**(最小 `k`),避免一次刷屏。
---
## 10. 边界与安全
| 场景 | 处理 |
|---|---|
| 新用户 | `created_at` 作活跃基线(恒非空)→ 注册即"第 1 日活跃";注册后连续 15 天无 home_view/比价/领券 才清 |
| 在途提现 | 提现申请时现金已扣入 `WithdrawOrder`,当前余额已不含在途;只清当前余额、不动提现单。提现失败退款到已清账户 = 用户的钱,正常 |
| 与 `daily_auto_exchange` 并存 | 各自逐用户幂等;金币多已日结折现金,三桶全清正好覆盖 |
| 时区/日界 | 统一北京(`rewards.cn_today()`/`CN_TZ`);**清零/预警按北京自然日 0 点对齐**(末次活跃记为第 1 日 → 第 16 日 0 点清零,见 §4),非滚动 24h;流水 `created_at` 沿用北京 wall-clock naive |
| 误清防护 | worker 常驻默认 **dry-run**`ENABLED=false` 只记审计名单、不动钱);看准名单再置 `true` 真清(§13 |
---
## 11. 前端依赖:`home_view` 埋点(跨仓 — Android
- **Android 端**`shaguabijia-app-android`)需在**首页可见**`onResume`/Tab 切入)时,向现有 `POST /api/v1/analytics/events` 批量上报里加一条 `event=<首页可见事件名>`(名称明天加埋点时定,暂记 `"home_view"` 的事件,**携带登录后的 `user_id`**。
- 客户端按会话/前台去重即可(服务端只取 `max(created_at)`,多报无害)。
- **上线顺序依赖**`home_view` 全量覆盖前,"进首页"信号缺失,只有比价/领券能推进活跃、其余落到 `created_at` 基线("只开首页不操作"且注册满 15 天的用户会被误清)—— 故**开真清(`ENABLED=true`)必须待 `home_view` 铺满后再开**(§13);dry-run 只记名单不动钱、可先开着看。
---
## 12. admin 重构范围与影响(R4
- `app/admin/repositories/queries.py`:删本地 `_ACTIVE_EVENTS`/`_last_active_parts()`,改用 `activity.py` 的常量与子查询构造;`list_users``greatest(...)` 排序/筛选、`_attach_last_active` 均改走共享构造器。
- `app/admin/repositories/stats.py``COMPARE_START_EVENT`/`COUPON_START_EVENT`/活跃用户集(`:138-146`)改用共享常量与口径。
- **行为变化(预期内、需产品知会)**admin 的"最近活跃 / DAU"口径变化——**移除 `last_login_at`(登录不再计为活跃)、以 `created_at` 为基线、纳入 `home_view`**。net`home_view` 铺满后更准(真正把"开首页"算进活跃);铺满前"只登录不操作"的用户活跃度会下降。
- **回归底线**:现有 admin 用户列表 / stats 测试按新口径**更新预期**last_login_at 移除 + created_at 基线 + home_view 纳入);非活跃口径部分行为不变。
---
## 13. 灰度与上线顺序(安全优先)
1. **后端先行**:合入共享模块 + 两表 + worker + 通知器,`INACTIVITY_RESET_ENABLED=False`;活跃口径以 `created_at` 为非空基线、**不含 last_login_at**。
2. **Android 发版**:上报 `home_view`;观察 analytics 覆盖率。
3. **dry-run 灰度(默认即是)**`INACTIVITY_RESET_ENABLED=False` 时 worker 常驻只写审计名单(`reason=inactive_Nd_dryrun`)、不动钱、不预警;核对名单准确。
4. **开真清**:确认无误后置 `INACTIVITY_RESET_ENABLED=True`(转为真清 + 预警)。
5. **收尾/监控**:持续观察 `home_view` 覆盖率与预警/清零名单;发现"活跃却被判不活跃"的漏报即回查埋点覆盖(口径已不含 last_login_at,登录不再兜底)。
---
## 14. 测试计划
- **活跃口径(共享模块)**`home_view`/比价/领券 各单独命中都算活跃;**纯登录不算**;无信号用户以 `created_at` 计;`max` 取最新;naive/aware 混算不崩。
- **admin 回归**:用户列表 / stats 按新口径更新预期(移除 last_login_at + created_at 基线 + home_view)。
- **不活跃判定**`last_active` 分别 `<15d / =15d / >15d` × 有/无余额 的命中矩阵。
- **清零**:三桶归零;`inactivity_reset_log` 清前值正确;三条流水 `biz_type=inactivity_reset``balance_after=0``ref_id=log.id``total_coin_earned` 不变。
- **预警**:命中窗口调 notifier + 写 `notification_log`;同 streak 不重推;回归后 `last_active` 前移可再次预警;漏跑补发最紧急档。
- **worker**:常驻;`ENABLED=false` 走 dry-run(只记审计名单、不清、不预警);文件锁互斥;逐用户失败隔离(一个抛错不影响其余,`failed` 计数);重复跑幂等。
- **配置**`INACTIVITY_WARN_DAYS_BEFORE` 解析(含空串=不推);`RESET_DAYS`/`RUN_HOUR` 生效。
- 沿用 `tests/conftest.py`(临时 SQLite、`RATE_LIMIT_ENABLED=false`);外部通知 monkeypatch。
---
## 15. 未来工作
- 接真实 `JPushNotifier`(需用户级 `registration_id` 覆盖 + JPush push API/ `SmsNotifier`
- 如需 admin 后台可视化:不活跃/预警/清零名单与历史查询接口。
- 如量级增长导致每日 join 扫描变慢:再考虑物化 `last_active_at`(当前每日一次可接受)。
---
## 附:涉及文件清单
**新增**
- `app/repositories/activity.py` — 活跃口径唯一真源
- `app/models/inactivity_reset_log.py` — 审计表
- `app/models/inactivity_notification_log.py` — 预警/占位表
- `app/core/inactivity_reset_worker.py` — 每日 worker(仿 daily_exchange_worker
- `app/integrations/notifier.py` — 通知器协议 + `LogNotifier`(真实 JPush/短信后续同层扩展)
- `alembic/versions/<...>_add_inactivity_tables.py` — 建两表迁移
- `docs/database/inactivity_reset_log.md` / `inactivity_notification_log.md` — 表字典(随实现补)
- 对应 `tests/test_inactivity_reset.py`
**改动**
- `app/models/__init__.py` — 注册两模型
- `app/core/config.py``INACTIVITY_*` 配置
- `app/main.py` — lifespan 接线 start/stop worker
- `app/admin/repositories/queries.py``stats.py` — 改用 `activity.py`(§12
+139
View File
@@ -0,0 +1,139 @@
"""人工验证用:按「金币/现金/邀请」排列组合 + 活跃/新用户对照,造一批账号。
用法(仓库根目录,venv 解释器):
.venv/Scripts/python.exe scripts/seed_inactivity_cases.py # 造号(会先清掉上次 vcase*)
.venv/Scripts/python.exe scripts/seed_inactivity_cases.py --clean # 只清理,不造
配合默认配置 INACTIVITY_RESET_DAYS=15 / INACTIVITY_WARN_DAYS_BEFORE=7,2 验证
造完把 worker 打开( README/对话里的 .env),启动服务即会在 RUN_HOUR 后跑一轮
worker 清零针对**库里所有**符合条件的用户,不止 vcase*dev 库里若有其它"老且有余额、
无近期活跃事件"的用户,也会被一起清。要干净验证建议用一个空/副本 dev 库。
"""
from __future__ import annotations
import os
import sys
from datetime import UTC, datetime, timedelta
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from sqlalchemy import delete, select # noqa: E402
from app.db.session import SessionLocal # noqa: E402
from app.models.analytics_event import AnalyticsEvent # noqa: E402
from app.models.inactivity import ( # noqa: E402
InactivityNotificationLog,
InactivityResetLog,
)
from app.models.user import User # noqa: E402
from app.models.wallet import ( # noqa: E402
CashTransaction,
CoinAccount,
CoinTransaction,
InviteCashTransaction,
)
from app.repositories import wallet as wallet_repo # noqa: E402
MARK = "vcase" # username 前缀,用于清理
# label, 创建于N天前, coin, cash, invite, 近期事件(N天前)or None, 预期
CASES = [
("1 三桶全有", 30, 100, 200, 300, None, "清 coin+cash;invite=300 保留;审计1行+2流水"),
("2 金币+现金", 30, 100, 200, 0, None, "清 coin+cash;审计1行+2流水"),
("3 金币+邀请", 30, 100, 0, 300, None, "清 coin;invite=300 保留;审计1行+1流水"),
("4 现金+邀请", 30, 0, 200, 300, None, "清 cash;invite=300 保留;审计1行+1流水"),
("5 只有金币", 30, 100, 0, 0, None, "清 coin;审计1行+1流水"),
("6 只有现金", 30, 0, 200, 0, None, "清 cash;审计1行+1流水"),
("7 只有邀请(红线)", 30, 0, 0, 300, None, "不选中/不清/无审计/无流水;invite=300 原封"),
("8 预警窗(10天)", 10, 50, 60, 70, None, "不清;发 T-7 预警;notification_log 1行;余额不动"),
("9 活跃兜底", 30, 100, 200, 300, 1, "昨日 home_view→last_active 近→不清不警"),
("10 新用户(3天)", 3, 100, 200, 0, None, "created_at 近→不清不警"),
]
def _mark_uids(db) -> list[int]:
return list(db.execute(select(User.id).where(User.username.like(f"{MARK}%"))).scalars())
def clean(db) -> int:
uids = _mark_uids(db)
if uids:
for model in (
InactivityResetLog, InactivityNotificationLog,
CoinTransaction, CashTransaction, InviteCashTransaction,
AnalyticsEvent, CoinAccount,
):
db.execute(delete(model).where(model.user_id.in_(uids)))
db.execute(delete(User).where(User.id.in_(uids)))
db.commit()
return len(uids)
def seed(db) -> None:
now = datetime.now(UTC)
print(f"{'#':>3} {'uid':>5} {'案例':<16} {'coin/cash/invite':<18} {'创建':<7} 预期")
for i, (label, days_ago, coin, cash, invite, ev_days, expected) in enumerate(CASES, 1):
u = User(
phone=f"seed_tmp_{i}", username=f"{MARK}{i}", status="active",
created_at=now - timedelta(days=days_ago),
last_login_at=now, # 登录很新——但登录不算活跃,清零该发生照发生
)
db.add(u)
db.flush() # 拿自增 id
u.phone = f"1{u.id:010d}" # 用全局唯一 id 拼 "100…" 段手机号,dev 库里绝不撞
acc = wallet_repo.get_or_create_account(db, u.id, commit=False)
acc.coin_balance, acc.cash_balance_cents, acc.invite_cash_balance_cents = coin, cash, invite
acc.total_coin_earned = coin
if ev_days is not None:
db.add(AnalyticsEvent( # 首页可见 = event=show + page=home
event="show", page="home", device_id=MARK, user_id=u.id,
client_ts=0, created_at=now - timedelta(days=ev_days),
))
db.flush()
print(f"{i:>3} {u.id:>5} {label:<16} {f'{coin}/{cash}/{invite}':<18} {f'{days_ago}天前':<7} {expected}")
db.commit()
def check(db) -> None:
"""worker 跑完后:打印每个 vcase 账号的当前三桶余额 + 是否有审计/预警行。"""
rows = db.execute(
select(User.id, User.username).where(User.username.like(f"{MARK}%")).order_by(User.id)
).all()
if not rows:
print("没有 vcase* 账号(先跑一次不带参数造号)")
return
print(f"{'uid':>5} {'账号':<8} {'coin/cash/invite(现在)':<24} {'审计':<5} 预警")
for uid, uname in rows:
acc = db.get(CoinAccount, uid)
bal = f"{acc.coin_balance}/{acc.cash_balance_cents}/{acc.invite_cash_balance_cents}" if acc else ""
has_reset = db.execute(
select(InactivityResetLog.id).where(InactivityResetLog.user_id == uid).limit(1)
).first()
stages = db.execute(
select(InactivityNotificationLog.stage).where(InactivityNotificationLog.user_id == uid)
).scalars().all()
warn = ",".join(f"T-{s}" for s in stages) if stages else ""
print(f"{uid:>5} {uname:<8} {bal:<24} {'' if has_reset else '':<5} {warn}")
def main() -> None:
db = SessionLocal()
try:
if "--check" in sys.argv:
check(db)
return
removed = clean(db)
if removed:
print(f"已清理上次 {removed}{MARK}* 账号")
if "--clean" in sys.argv:
return
seed(db)
print("\n造号完成。打开 worker(INACTIVITY_RESET_ENABLED=true, RUN_HOUR=17)后启动服务,"
"≥17:00 首个 tick 即跑一轮。验完 `--clean` 清理。")
finally:
db.close()
if __name__ == "__main__":
main()
-160
View File
@@ -1,160 +0,0 @@
"""会话存档【独立进程】轮询 poller —— 单独 systemd 服务跑, 与 app-server 隔离。
为什么独立进程:WeWorkFinanceSdk Go 编译的 c-shared , 自带 Go runtime嵌进 app-server(uvicorn)
后台线程里跑会 segfault把主进程带崩(2026-07-17 实测 core-dump 5s 循环)独立进程让 Go runtime
独占进程 + 主线程, 崩了 systemd 只重启本 poller不动 app-server拉到美团卡片(weapp)/截图(image)
POST app-server 内部端点 /internal/wx-finance/pending 打比价信号(下游心跳弹窗比价 完全复用)
手动跑: cd /opt/shaguabijia-app-server && .venv/bin/python scripts/wx_finance_poller.py
生产: deploy/wx-finance-poller.service 常驻(systemctl enable --now wx-finance-poller)
"""
from __future__ import annotations
import json
import logging
import signal
import time
from pathlib import Path
import httpx
from app.core.config import settings
from app.core.logging import setup_logging
from app.integrations.wx_finance_sdk import WxFinanceError, WxFinanceSdk
logger = logging.getLogger("shagua.wx_finance_poller")
_stop = False
def _on_signal(_signum: int, _frame: object) -> None:
global _stop
_stop = True
def _load_seq() -> int:
try:
return int(Path(settings.WX_FINANCE_SEQ_FILE).read_text().strip() or "0")
except Exception:
return 0
def _save_seq(seq: int) -> None:
try:
p = Path(settings.WX_FINANCE_SEQ_FILE)
p.parent.mkdir(parents=True, exist_ok=True)
p.write_text(str(seq))
except Exception:
logger.exception("seq 持久化失败")
def _notify(client: httpx.Client, source: str, kind: str) -> None:
"""POST 内部端点让 app-server 打信号(set_pending 在 app-server 进程内存)。"""
url = f"{settings.WX_FINANCE_INTERNAL_URL.rstrip('/')}/internal/wx-finance/pending"
try:
resp = client.post(
url,
json={"source": source, "kind": kind},
headers={"X-Internal-Secret": settings.INTERNAL_API_SECRET},
timeout=10,
)
logger.info("notify app-server (%s) → %s", kind, resp.status_code)
except Exception:
logger.exception("notify app-server 失败")
def _resolve_source(msg: dict) -> str:
"""从 weapp 小程序卡判源平台代号(= app-server _SOURCE_PACKAGES 的 key)。
会话存档 weapp 结构里 username(gh_id)/appid/pagepath/title 任一含平台特征即判定:
先按关键词启发式(覆盖大多数);认不出 默认 meituan(过渡期不阻断)+ WARNING,
并把整个 weapp 结构打进日志,便于照真实 username/appid 精确补规则
"""
weapp = msg.get("weapp") or {}
blob = json.dumps(weapp, ensure_ascii=False).lower()
# 排障日志:打全 weapp 结构(顺带 msg 顶层 key,防结构不在 weapp 下),便于补/改判定规则
logger.info("weapp 判源: msg_keys=%s weapp=%s", list(msg.keys()), blob[:1500])
if any(k in blob for k in ("jingdong", "jd.com", "jddj", "京东")):
return "jd"
if any(k in blob for k in ("meituan", "sankuai", "美团")):
return "meituan"
logger.warning("weapp 源平台未识别, 暂默认 meituan;见上一行结构日志补 _resolve_source 规则")
return "meituan"
def _handle(client: httpx.Client, msg: dict) -> None:
"""明文消息:外部用户(from≠接收成员)发来的 weapp/image → 通知 app-server 打信号。"""
msgtype = msg.get("msgtype") or ""
frm = msg.get("from") or ""
receiver = settings.WX_FINANCE_RECEIVER_USERID
if receiver and frm == receiver:
return # 成员自己发的, 跳过
if msgtype not in ("weapp", "image"):
return
# weapp 卡片按小程序判源平台;image(截图)判不出平台,沿用 meituan。
source = _resolve_source(msg) if msgtype == "weapp" else "meituan"
logger.info("命中触发 from=%s type=%s source=%s", frm, msgtype, source)
# 归档延迟排查: msgtime 是腾讯给的消息发送时刻(毫秒), 与 poller 此刻之差
# ≈ 企业微信会话存档的归档延迟(+ 本地 ≤0.5s 轮询间隔), 用来隔离"是我们慢还是归档慢"。
try:
_msgtime_ms = float(msg.get("msgtime") or 0)
if _msgtime_ms > 0:
logger.info("归档延迟(msgtime→poller取到) ≈ %.2fs", time.time() - _msgtime_ms / 1000.0)
except Exception:
pass
_notify(client, source, msgtype)
def main() -> None:
setup_logging(debug=settings.APP_DEBUG)
if not settings.wx_finance_configured:
logger.warning("WX_FINANCE 未配齐(ENABLED/corpid/存档Secret), poller 退出")
return
if not settings.INTERNAL_API_SECRET:
logger.error("INTERNAL_API_SECRET 未配置, poller 无法通知 app-server, 退出")
return
signal.signal(signal.SIGTERM, _on_signal)
signal.signal(signal.SIGINT, _on_signal)
sdk = WxFinanceSdk(
settings.WX_FINANCE_SDK_PATH,
settings.WX_KF_CORP_ID,
settings.WX_FINANCE_SECRET,
Path(settings.WX_FINANCE_PRIVATE_KEY_PATH).read_bytes(),
)
seq = _load_seq()
logger.info(
"wx_finance poller 启动, 从 seq=%d 轮询, 间隔 %ss", seq, settings.WX_FINANCE_POLL_INTERVAL_SEC
)
with httpx.Client() as client:
while not _stop:
try:
msgs = sdk.get_chat_data(seq, limit=1000)
if msgs:
logger.info("拉到 %d 条(seq>%d)", len(msgs), seq)
for item in msgs:
try:
plain = sdk.decrypt(item["encrypt_random_key"], item["encrypt_chat_msg"])
_handle(client, plain)
except Exception:
logger.exception("解密/处理单条失败 seq=%s", item.get("seq"))
seq = max(seq, int(item.get("seq", seq)))
if msgs:
_save_seq(seq)
except WxFinanceError:
logger.exception("GetChatData 出错")
except Exception:
logger.exception("轮询异常")
# 可打断的 sleep:按 ≤0.5s 分片, 支持亚秒间隔(如 0.5s), 收到 SIGTERM 尽快退
interval = max(0.1, settings.WX_FINANCE_POLL_INTERVAL_SEC)
step = min(0.5, interval)
slept = 0.0
while slept < interval and not _stop:
time.sleep(step)
slept += step
sdk.close()
logger.info("wx_finance poller 已停止")
if __name__ == "__main__":
main()
-45
View File
@@ -1,45 +0,0 @@
"""会话存档 SDK 连通性探针 —— 上 worker 前,在【云服务器】单独验证整条链路是否打通。
/opt/shaguabijia-app-server 下跑: .venv/bin/python scripts/wx_finance_probe.py
依次验证:加载 .so NewSdk/Init(corpid+存档Secret+可信IP) GetChatData(0) 试解第一条
只读不写任何比价信号任一步报错都会明确指出卡在哪(便于逐项排:.so 缺依赖 / Init 失败 /
可信IP 没放行 / 私钥不匹配)全绿了再把 WX_FINANCE_ENABLED true worker
"""
from __future__ import annotations
from pathlib import Path
from app.core.config import settings
from app.integrations.wx_finance_sdk import WxFinanceSdk
def main() -> None:
print(f"[cfg] sdk_path = {settings.WX_FINANCE_SDK_PATH}")
print(f"[cfg] corpid = {settings.WX_KF_CORP_ID}")
print(f"[cfg] secret_set = {bool(settings.WX_FINANCE_SECRET)}")
print(f"[cfg] privkey = {settings.WX_FINANCE_PRIVATE_KEY_PATH}")
sdk = WxFinanceSdk(
settings.WX_FINANCE_SDK_PATH,
settings.WX_KF_CORP_ID,
settings.WX_FINANCE_SECRET,
Path(settings.WX_FINANCE_PRIVATE_KEY_PATH).read_bytes(),
)
print("[1] 加载 .so + Init 成功")
msgs = sdk.get_chat_data(0, limit=100)
print(f"[2] GetChatData(0) 成功, 拉到 {len(msgs)}")
if msgs:
first = msgs[0]
print(f" 第一条 seq={first.get('seq')} publickey_ver={first.get('publickey_ver')}")
plain = sdk.decrypt(first["encrypt_random_key"], first["encrypt_chat_msg"])
print(f"[3] 解密成功, msgtype={plain.get('msgtype')} from={plain.get('from')}")
else:
print("[3] 暂无历史消息(先让用户加接收成员好友、发条消息, 再跑一次)")
sdk.close()
print("探针完成:链路全通 ✓")
if __name__ == "__main__":
main()
+61
View File
@@ -107,6 +107,67 @@ def test_sms_send_device_ip_rate_limit(client, monkeypatch) -> None:
assert r.status_code == 200, r.text
def test_sms_send_cooldown_reject_not_counted(client, monkeypatch) -> None:
"""发码额度只算「成功发码」:被单号 60s 冷却挡下的重发(429)不占设备额度。
做法:同号狂发只成功 1 其余被冷却挡下;把小时额度设 2,证明换号后仍能再成功发 1
若冷却重发也计数,额度早被那几次耗尽"""
from app.api.v1 import auth
from app.core import ratelimit
monkeypatch.setattr(ratelimit.settings, "RATE_LIMIT_ENABLED", True)
monkeypatch.setattr(auth, "SMS_SEND_MAX_PER_HOUR_PER_DEVICE", 2)
ratelimit._buckets.clear()
device = "dev-cooldown"
phone_a = "13710137000"
# 首发成功(小时闸计 1/2)
assert client.post(
"/api/v1/auth/sms/send", json={"phone": phone_a, "device_id": device}
).status_code == 200
# 同号连发 3 次:都被单号 60s 冷却挡下 → 429,且**不占**设备额度
for _ in range(3):
r = client.post(
"/api/v1/auth/sms/send", json={"phone": phone_a, "device_id": device}
)
assert r.status_code == 429, r.text
# 换号再发:设备额度只用了 1/2(冷却那几次没算)→ 仍放行(计到 2/2)
assert client.post(
"/api/v1/auth/sms/send", json={"phone": "13710137001", "device_id": device}
).status_code == 200
# 又换号:此时小时闸已 2/2 → 429(反证成功发码确实各计了 1)
r = client.post(
"/api/v1/auth/sms/send", json={"phone": "13710137002", "device_id": device}
)
assert r.status_code == 429, r.text
def test_sms_send_daily_cap(client, monkeypatch) -> None:
"""每天发码上限(设备 + IP):成功发码累计到日上限即 429(用不同手机号绕开单号冷却)。
抬高小时闸单独测日闸;超限文案含今日以便前端提示明天再来"""
from app.api.v1 import auth
from app.core import ratelimit
monkeypatch.setattr(ratelimit.settings, "RATE_LIMIT_ENABLED", True)
monkeypatch.setattr(auth, "SMS_SEND_MAX_PER_HOUR_PER_DEVICE", 100) # 抬高小时闸,不干扰
monkeypatch.setattr(auth, "SMS_SEND_MAX_PER_DAY_PER_DEVICE", 3)
ratelimit._buckets.clear()
device = "dev-daily"
for i in range(3):
r = client.post(
"/api/v1/auth/sms/send",
json={"phone": f"13720137{i:03d}", "device_id": device},
)
assert r.status_code == 200, f"{i + 1} 次应放行: {r.text}"
# 第 4 次:同设备同 IP 当日超限 → 429
r = client.post(
"/api/v1/auth/sms/send",
json={"phone": "13720137999", "device_id": device},
)
assert r.status_code == 429, r.text
assert "今日" in r.json()["detail"]
def test_sms_login_device_ip_rate_limit(client, monkeypatch) -> None:
"""防刷:同一设备(device_id) + 同一 IP 每小时最多 SMS_LOGIN_MAX_PER_HOUR 次登录尝试,超出 429。
conftest 默认 RATE_LIMIT_ENABLED=false(内存计数跨用例累加),本用例临时打开并清空计数隔离"""
+478
View File
@@ -0,0 +1,478 @@
"""15 天不活跃清零:模型 / 活跃口径 / 清零 / 预警 / 配置 / worker。"""
from __future__ import annotations
from datetime import date, datetime, timedelta, timezone
import pytest
from sqlalchemy import delete, select, update
from app.db.session import SessionLocal
from app.models.inactivity import InactivityNotificationLog, InactivityResetLog
from app.repositories import activity
def test_reset_and_notification_models_persist() -> None:
db = SessionLocal()
try:
db.add(InactivityResetLog(
user_id=1, coin_balance_before=10, cash_balance_cents_before=20,
invite_cash_balance_cents_before=30,
last_active_at=datetime(2026, 1, 1, tzinfo=timezone.utc),
inactive_days=15, reason="inactive_15d",
))
db.add(InactivityNotificationLog(
user_id=1, stage=7, inactive_days=8, coin_balance=10,
cash_balance_cents=20, invite_cash_balance_cents=30,
channel="log", status="placeholder",
))
db.commit()
r = db.execute(select(InactivityResetLog).where(InactivityResetLog.user_id == 1)).scalar_one()
assert r.reason == "inactive_15d" and r.reset_at is not None
n = db.execute(select(InactivityNotificationLog).where(InactivityNotificationLog.user_id == 1)).scalar_one()
assert n.stage == 7 and n.created_at is not None
finally:
db.rollback()
db.close()
def test_reset_cutoff_is_cn_midnight_of_today_minus_days_minus_1() -> None:
# RESET_DAYS=15, today=1/20 → cutoff = 北京 00:00 of 1/6 = 1/5 16:00 UTC
cutoff = activity.reset_cutoff(15, today=date(2026, 1, 20))
assert cutoff == datetime(2026, 1, 5, 16, 0, tzinfo=timezone.utc)
def test_active_event_constants() -> None:
# 首页可见 = event=show + page=home 组合,不在纯 event 名集合里
assert activity.HOME_VIEW_EVENT == "show" and activity.HOME_VIEW_PAGE == "home"
assert activity.HOME_VIEW_EVENT not in activity.ACTIVE_EVENTS
assert "real_compare_start" in activity.ACTIVE_EVENTS
assert "real_coupon_start" in activity.ACTIVE_EVENTS
assert activity.ACTIVE_ENGAGE_TYPE == "claim_started"
def test_as_utc_normalizes() -> None:
assert activity.as_utc(datetime(2026, 1, 1)) == datetime(2026, 1, 1, tzinfo=timezone.utc)
cn = datetime(2026, 1, 1, tzinfo=activity.CN_TZ) # 北京 0 点 = 前一天 16:00 UTC
assert activity.as_utc(cn) == datetime(2025, 12, 31, 16, 0, tzinfo=timezone.utc)
from app.core.rewards import CN_TZ
from app.models.analytics_event import AnalyticsEvent
from app.models.coupon_state import CouponPromptEngagement
from app.models.user import User
from app.models.wallet import CoinAccount
from app.repositories import wallet as wallet_repo
_PHONE_SEQ = [0]
@pytest.fixture(autouse=True)
def _isolate_inactivity_state():
"""本文件的测试都做全表扫描 + 全局计数,而 SQLite 测试库 session 级共享、无逐用例回滚
(commit 后的 rollback no-op),故先把可能泄漏的余额清零 + 清掉活跃事件/审计行,
保证每个用例干净起步不删 User(零余额用户不会被扫描选中,避免跨文件/外键影响)"""
db = SessionLocal()
try:
db.execute(update(CoinAccount).values(
coin_balance=0, cash_balance_cents=0, invite_cash_balance_cents=0))
for model in (AnalyticsEvent, CouponPromptEngagement,
InactivityResetLog, InactivityNotificationLog):
db.execute(delete(model))
db.commit()
finally:
db.close()
yield
def _new_user(db, *, created_at, coin=0, cash=0, invite=0) -> int:
"""直接建一个 User + CoinAccount,created_at 可控。返回 user_id。"""
_PHONE_SEQ[0] += 1
# 199 前缀 + 递增序号:共享测试库跨文件累积用户,别的文件用固定手机号(如 test_admin_write
# 的 13900000001..),这里用没人用的 199 段避免撞 user.phone / username 的 UNIQUE。
u = User(phone=f"199{_PHONE_SEQ[0]:08d}", created_at=created_at,
last_login_at=created_at, status="active",
username=f"inact{_PHONE_SEQ[0]}")
db.add(u)
db.flush()
acc = wallet_repo.get_or_create_account(db, u.id, commit=False)
acc.coin_balance, acc.cash_balance_cents, acc.invite_cash_balance_cents = coin, cash, invite
acc.total_coin_earned = coin
db.flush()
return u.id
def _add_event(db, user_id, event, when: datetime, page=None) -> None:
db.add(AnalyticsEvent(event=event, device_id="d", user_id=user_id, client_ts=0,
created_at=when, page=page))
def _add_engage(db, user_id, when: datetime, engage_type="claim_started") -> None:
db.add(CouponPromptEngagement(device_id=f"dev{user_id}", package="p", user_id=user_id,
engage_date=when.date(), engage_type=engage_type, created_at=when))
def test_last_active_expr_takes_max_of_baseline_and_events() -> None:
from sqlalchemy import select
db = SessionLocal()
try:
base = datetime(2026, 1, 1, tzinfo=timezone.utc)
uid = _new_user(db, created_at=base, coin=5)
_add_event(db, uid, "real_compare_start", datetime(2026, 1, 10, tzinfo=timezone.utc))
db.commit()
ev_sub, eng_sub = activity.last_active_subqueries(db)
dialect = db.get_bind().dialect.name
expr = activity.last_active_expr(User.created_at, ev_sub, eng_sub, dialect)
stmt = (select(expr).select_from(User)
.outerjoin(ev_sub, ev_sub.c.user_id == User.id)
.outerjoin(eng_sub, eng_sub.c.user_id == User.id)
.where(User.id == uid))
got = activity.norm_utc(db.execute(stmt).scalar_one())
assert got == datetime(2026, 1, 10, tzinfo=timezone.utc) # 事件 > 基线
finally:
db.rollback()
db.close()
def test_home_signal_uses_show_event_on_home_page() -> None:
"""首页可见活跃口径 = event=show + page=home 组合;show 但非 home 页不算活跃。"""
from sqlalchemy import select
db = SessionLocal()
try:
base = datetime(2026, 1, 1, tzinfo=timezone.utc)
seen = _new_user(db, created_at=base) # show/home → 活跃
_add_event(db, seen, "show", datetime(2026, 1, 10, tzinfo=timezone.utc), page="home")
other = _new_user(db, created_at=base) # show/其他页 → 不算活跃
_add_event(db, other, "show", datetime(2026, 1, 10, tzinfo=timezone.utc), page="coupon")
db.commit()
ev_sub, eng_sub = activity.last_active_subqueries(db)
dialect = db.get_bind().dialect.name
expr = activity.last_active_expr(User.created_at, ev_sub, eng_sub, dialect)
def last_active(uid):
stmt = (select(expr).select_from(User)
.outerjoin(ev_sub, ev_sub.c.user_id == User.id)
.outerjoin(eng_sub, eng_sub.c.user_id == User.id)
.where(User.id == uid))
return activity.norm_utc(db.execute(stmt).scalar_one())
assert last_active(seen) == datetime(2026, 1, 10, tzinfo=timezone.utc) # show/home 算
assert last_active(other) == base # show/其他页 不算
finally:
db.rollback()
db.close()
def test_inactivity_warn_stages_parsing() -> None:
from app.core.config import Settings
s = Settings(INACTIVITY_WARN_DAYS_BEFORE="7,2", INACTIVITY_RESET_DAYS=15)
assert s.inactivity_warn_stages == [7, 2] # 降序去重
s2 = Settings(INACTIVITY_WARN_DAYS_BEFORE="", INACTIVITY_RESET_DAYS=15)
assert s2.inactivity_warn_stages == [] # 空=不推
s3 = Settings(INACTIVITY_WARN_DAYS_BEFORE="2,20,7,2", INACTIVITY_RESET_DAYS=15)
assert s3.inactivity_warn_stages == [7, 2] # 去重 + 丢弃 >=RESET_DAYS(20)
def test_log_notifier_returns_placeholder(caplog) -> None:
from app.integrations.notifier import LogNotifier, get_notifier
n = get_notifier("log")
assert isinstance(n, LogNotifier) and n.channel == "log"
status = n.warn(user_id=1, coin=10, cash_cents=20, stage=7, days_until_reset=8)
assert status == "placeholder"
# 未实现通道回退 LogNotifier(占位)
assert get_notifier("jpush").channel == "log"
def test_run_reset_clears_coin_and_cash_but_preserves_invite_cash() -> None:
from sqlalchemy import select
from app.models.wallet import CoinAccount, CoinTransaction, CashTransaction, InviteCashTransaction
from app.repositories import inactivity
db = SessionLocal()
try:
today = date(2026, 2, 1)
# 末次活跃 = created_at 基线 = 1/10(距 today 22 天 → 应清)
old = _new_user(db, created_at=datetime(2026, 1, 10, tzinfo=timezone.utc),
coin=100, cash=200, invite=300)
# 活跃用户:昨天有 home_view → 不清
fresh = _new_user(db, created_at=datetime(2026, 1, 1, tzinfo=timezone.utc), coin=50)
_add_event(db, fresh, "show", datetime(2026, 1, 31, tzinfo=timezone.utc), page="home")
db.commit()
stats = inactivity.run_reset_once(db, reset_days=15, today=today)
assert stats["cleared"] == 1 and stats["failed"] == 0
acc = db.get(CoinAccount, old)
# 金币 + 折算现金清零;邀请现金是产品红线,原封不动(见 wallet.CoinAccount 注释)
assert (acc.coin_balance, acc.cash_balance_cents) == (0, 0)
assert acc.invite_cash_balance_cents == 300
assert acc.total_coin_earned == 100 # 历史累计不动
log = db.execute(select(InactivityResetLog).where(InactivityResetLog.user_id == old)).scalar_one()
# 审计仍快照三桶余额(邀请现金记为"清零时仍保留"的余额,便于纠纷排查)
assert (log.coin_balance_before, log.cash_balance_cents_before,
log.invite_cash_balance_cents_before) == (100, 200, 300)
assert log.inactive_days == 22 and log.reason == "inactive_15d"
ct = db.execute(select(CoinTransaction).where(
CoinTransaction.user_id == old, CoinTransaction.biz_type == "inactivity_reset")).scalar_one()
assert ct.amount == -100 and ct.balance_after == 0 and ct.ref_id == str(log.id)
assert db.execute(select(CashTransaction).where(
CashTransaction.user_id == old, CashTransaction.biz_type == "inactivity_reset")).scalar_one().amount_cents == -200
# 关键:不写邀请现金流水(邀请现金不清)
assert db.execute(select(InviteCashTransaction).where(
InviteCashTransaction.user_id == old,
InviteCashTransaction.biz_type == "inactivity_reset")).first() is None
# 活跃用户不动;再跑一次幂等(coin+cash 已 0、邀请现金不算候选 → 不再匹配)
assert db.get(CoinAccount, fresh).coin_balance == 50
assert inactivity.run_reset_once(db, reset_days=15, today=today)["cleared"] == 0
finally:
db.rollback()
db.close()
def test_user_with_only_invite_cash_is_not_cleared() -> None:
"""只有邀请现金余额的久不活跃用户:邀请现金是产品红线,不清 → 根本不该被选中。"""
from app.models.wallet import CoinAccount
from app.repositories import inactivity
db = SessionLocal()
try:
today = date(2026, 2, 1)
uid = _new_user(db, created_at=datetime(2026, 1, 10, tzinfo=timezone.utc),
coin=0, cash=0, invite=500)
db.commit()
stats = inactivity.run_reset_once(db, reset_days=15, today=today)
assert stats["cleared"] == 0
assert db.get(CoinAccount, uid).invite_cash_balance_cents == 500 # 原封不动
finally:
db.rollback()
db.close()
def test_run_warn_picks_stage_and_dedups_within_streak() -> None:
from app.integrations.notifier import LogNotifier
from app.repositories import inactivity
db = SessionLocal()
try:
today = date(2026, 2, 1)
# 末次活跃 1/22(距 today 10 天)→ 档 7 命中(idays>=8),档 2 未到(需>=13)
uid = _new_user(db, created_at=datetime(2026, 1, 22, tzinfo=timezone.utc), coin=100)
db.commit()
stats = inactivity.run_warn_once(db, LogNotifier(), reset_days=15, warn_stages=[7, 2], today=today)
assert stats["warned"] == 1
from sqlalchemy import select
rows = db.execute(select(InactivityNotificationLog).where(
InactivityNotificationLog.user_id == uid)).scalars().all()
assert len(rows) == 1 and rows[0].stage == 7 and rows[0].status == "placeholder"
assert rows[0].inactive_days == 10 and rows[0].coin_balance == 100
# 同一 streak 再跑 → 不重推
assert inactivity.run_warn_once(db, LogNotifier(), reset_days=15, warn_stages=[7, 2], today=today)["warned"] == 0
# 无余额用户不预警
_new_user(db, created_at=datetime(2026, 1, 22, tzinfo=timezone.utc), coin=0)
db.commit()
assert inactivity.run_warn_once(db, LogNotifier(), reset_days=15, warn_stages=[7, 2], today=today)["warned"] == 0
finally:
db.rollback()
db.close()
def test_run_once_warns_then_resets() -> None:
from app.integrations.notifier import LogNotifier
from app.models.wallet import CoinAccount
from app.repositories import inactivity
db = SessionLocal()
try:
today = date(2026, 2, 1)
warn_uid = _new_user(db, created_at=datetime(2026, 1, 22, tzinfo=timezone.utc), coin=10) # 10天→预警
clear_uid = _new_user(db, created_at=datetime(2026, 1, 5, tzinfo=timezone.utc), coin=10) # 27天→清零
db.commit()
stats = inactivity.run_once(db, notifier=LogNotifier(), reset_days=15, warn_stages=[7, 2], today=today)
assert stats["warned"] == 1 and stats["cleared"] == 1
assert db.get(CoinAccount, clear_uid).coin_balance == 0
assert db.get(CoinAccount, warn_uid).coin_balance == 10 # 预警不动钱
finally:
db.rollback()
db.close()
def test_worker_run_once_entry_dry_run(monkeypatch) -> None:
"""ENABLED=false(默认语义)→ worker 常驻但只记审计不清(dry_run = not ENABLED)。"""
from sqlalchemy import select
from app.core import inactivity_reset_worker as w
from app.core.config import settings
from app.models.wallet import CoinAccount
monkeypatch.setattr(settings, "INACTIVITY_RESET_ENABLED", False) # false = 只记审计
monkeypatch.setattr(settings, "INACTIVITY_RESET_DAYS", 15)
monkeypatch.setattr(settings, "INACTIVITY_WARN_DAYS_BEFORE", "")
monkeypatch.setattr(w, "_cn_today", lambda: date(2026, 2, 1))
db = SessionLocal()
try:
uid = _new_user(db, created_at=datetime(2026, 1, 1, tzinfo=timezone.utc), coin=100)
db.commit()
finally:
db.close()
w._run_once_entry()
db = SessionLocal()
try:
assert db.get(CoinAccount, uid).coin_balance == 100 # 没清
log = db.execute(select(InactivityResetLog).where(InactivityResetLog.user_id == uid)).scalar_one()
assert log.reason.endswith("dryrun") # 记了审计
finally:
db.close()
def test_worker_run_once_entry_executes(monkeypatch) -> None:
"""_run_once_entry 用真实 SessionLocal 跑一轮,总闸开时能清掉一个不活跃用户。"""
from app.core import inactivity_reset_worker as w
from app.core.config import settings
from app.models.wallet import CoinAccount
monkeypatch.setattr(settings, "INACTIVITY_RESET_ENABLED", True)
monkeypatch.setattr(settings, "INACTIVITY_RESET_DAYS", 15)
monkeypatch.setattr(settings, "INACTIVITY_WARN_DAYS_BEFORE", "") # 只测清零
# 固定"今天"避免依赖真实时钟
monkeypatch.setattr(w, "_cn_today", lambda: date(2026, 2, 1))
db = SessionLocal()
try:
uid = _new_user(db, created_at=datetime(2026, 1, 1, tzinfo=timezone.utc), coin=100)
db.commit()
finally:
db.close()
stats = w._run_once_entry()
assert stats["cleared"] >= 1
db = SessionLocal()
try:
assert db.get(CoinAccount, uid).coin_balance == 0
finally:
db.close()
def test_run_once_dry_run_records_audit_but_does_not_clear() -> None:
"""dry-run:只写审计(标 dryrun)、不动钱、不预警;重复跑不重复记(streak dedup)。"""
from sqlalchemy import select
from app.integrations.notifier import LogNotifier
from app.models.wallet import CoinAccount, CoinTransaction
from app.repositories import inactivity
db = SessionLocal()
try:
today = date(2026, 2, 1)
old = _new_user(db, created_at=datetime(2026, 1, 10, tzinfo=timezone.utc), coin=100, cash=200, invite=300)
warn_uid = _new_user(db, created_at=datetime(2026, 1, 22, tzinfo=timezone.utc), coin=50) # 预警窗
db.commit()
stats = inactivity.run_once(db, notifier=LogNotifier(), reset_days=15,
warn_stages=[7, 2], today=today, dry_run=True)
acc = db.get(CoinAccount, old)
assert (acc.coin_balance, acc.cash_balance_cents, acc.invite_cash_balance_cents) == (100, 200, 300) # 原封
log = db.execute(select(InactivityResetLog).where(InactivityResetLog.user_id == old)).scalar_one()
assert log.coin_balance_before == 100 and log.reason.endswith("dryrun") # 审计标 dryrun
assert db.execute(select(CoinTransaction).where(
CoinTransaction.user_id == old, CoinTransaction.biz_type == "inactivity_reset")).first() is None # 无流水
assert stats["warned"] == 0 # dry-run 不预警
assert db.execute(select(InactivityNotificationLog).where(
InactivityNotificationLog.user_id == warn_uid)).first() is None
assert stats["cleared"] == 1 # dry-run:cleared=记了几条
# 再跑一次 → 不重复记(dedup),余额仍原封
inactivity.run_once(db, notifier=LogNotifier(), reset_days=15, warn_stages=[7, 2], today=today, dry_run=True)
assert len(db.execute(select(InactivityResetLog).where(
InactivityResetLog.user_id == old)).scalars().all()) == 1
assert db.get(CoinAccount, old).coin_balance == 100
finally:
db.rollback()
db.close()
def test_admin_list_users_last_active_ignores_login() -> None:
"""admin 用户列表 last_active_at 改用共享口径:登录不算活跃(baseline=created_at)、只认活跃事件。"""
from app.admin.repositories import queries
db = SessionLocal()
try:
created = datetime(2026, 1, 1, tzinfo=timezone.utc)
uid = _new_user(db, created_at=created)
u = db.get(User, uid)
u.last_login_at = datetime(2026, 6, 1, tzinfo=timezone.utc) # 登录很新、但无任何活跃事件
db.commit()
phone = db.get(User, uid).phone
users, _cursor, _total = queries.list_users(db, phone=phone)
item = next(x for x in users if x.id == uid)
assert activity.norm_utc(item.last_active_at) == created # 登录不算 → last_active=created_at
finally:
db.rollback()
db.close()
def test_run_warn_isolates_notifier_failure_and_does_not_block_reset() -> None:
"""单用户通知器抛错:预警计 warn_failed、不外抛,且清零(reset)照常执行。"""
from app.models.wallet import CoinAccount
from app.repositories import inactivity
class BoomNotifier:
channel = "log"
def warn(self, *, user_id, coin, cash_cents, stage, days_until_reset) -> str:
raise RuntimeError("push service down")
db = SessionLocal()
try:
today = date(2026, 2, 1)
warn_uid = _new_user(db, created_at=datetime(2026, 1, 22, tzinfo=timezone.utc), coin=10) # 10天→预警
clear_uid = _new_user(db, created_at=datetime(2026, 1, 5, tzinfo=timezone.utc), coin=10) # 27天→清零
db.commit()
stats = inactivity.run_once(db, notifier=BoomNotifier(), reset_days=15,
warn_stages=[7, 2], today=today)
assert stats["warned"] == 0 and stats["warn_failed"] >= 1 # 预警失败被隔离
assert stats["cleared"] == 1 # 关键:清零没被阻塞
assert db.get(CoinAccount, clear_uid).coin_balance == 0
assert db.get(CoinAccount, warn_uid).coin_balance == 10 # 预警用户不动钱
# 预警失败已回滚,不留半条 notification_log
from sqlalchemy import select
assert db.execute(select(InactivityNotificationLog).where(
InactivityNotificationLog.user_id == warn_uid)).first() is None
finally:
db.rollback()
db.close()
def test_run_once_reset_runs_even_if_warn_phase_throws(monkeypatch) -> None:
"""预警整段异常(如候选查询失败)也绝不阻塞清零。"""
from app.integrations.notifier import LogNotifier
from app.models.wallet import CoinAccount
from app.repositories import inactivity
def boom(*a, **k):
raise RuntimeError("warn phase blew up")
monkeypatch.setattr(inactivity, "run_warn_once", boom)
db = SessionLocal()
try:
today = date(2026, 2, 1)
clear_uid = _new_user(db, created_at=datetime(2026, 1, 5, tzinfo=timezone.utc), coin=10)
db.commit()
stats = inactivity.run_once(db, notifier=LogNotifier(), reset_days=15,
warn_stages=[7, 2], today=today)
assert stats["cleared"] == 1
assert db.get(CoinAccount, clear_uid).coin_balance == 0
finally:
db.rollback()
db.close()
+234
View File
@@ -0,0 +1,234 @@
"""接口指标可观测(observe)单测:配置门槛 / 队列 / 中间件 / worker。
沿用仓库约定:TestClient + monkeypatch,绝不打真网络observe 默认关(conftest 未设
OBSERVE_*),需要开启的用例用 monkeypatch settings 单例属性
"""
from __future__ import annotations
import asyncio
import httpx
import pytest
from fastapi import FastAPI
from fastapi.testclient import TestClient
from app.core import observe, observe_worker
from app.core.config import settings
def test_observe_configured_requires_switch_and_creds(monkeypatch):
# 开关开 + endpoint(默认 localhost)+ user + password 齐全 → True
monkeypatch.setattr(settings, "OBSERVE_ENABLED", True)
monkeypatch.setattr(settings, "OBSERVE_USER", "u")
monkeypatch.setattr(settings, "OBSERVE_PASSWORD", "p")
assert settings.observe_configured is True
# 缺密码 → False
monkeypatch.setattr(settings, "OBSERVE_PASSWORD", "")
assert settings.observe_configured is False
# 缺用户名 → False
monkeypatch.setattr(settings, "OBSERVE_PASSWORD", "p")
monkeypatch.setattr(settings, "OBSERVE_USER", "")
assert settings.observe_configured is False
# 开关关 → False(即便凭证齐全)
monkeypatch.setattr(settings, "OBSERVE_USER", "u")
monkeypatch.setattr(settings, "OBSERVE_ENABLED", False)
assert settings.observe_configured is False
def test_record_event_enqueues(monkeypatch):
q = asyncio.Queue(maxsize=10)
monkeypatch.setattr(observe, "_queue", q)
observe.record_event({"route": "/x"})
assert q.get_nowait() == {"route": "/x"}
def test_record_event_drops_when_full(monkeypatch):
q = asyncio.Queue(maxsize=1)
monkeypatch.setattr(observe, "_queue", q)
monkeypatch.setattr(observe, "_dropped", 0)
observe.record_event({"n": 1}) # 占满
observe.record_event({"n": 2}) # 满 → 丢弃当前,不抛异常
assert observe.take_dropped() == 1
assert observe.take_dropped() == 0 # 取出后清零
assert q.get_nowait() == {"n": 1} # 保留的是先到的
def _make_probe_app() -> FastAPI:
"""独立最小 app:只挂中间件 + 两个无鉴权路由,不碰真业务 DB/auth。"""
app = FastAPI()
app.add_middleware(observe.RequestMetricsMiddleware)
@app.get("/things/{tid}")
def get_thing(tid: str):
return {"tid": tid}
@app.get("/health")
def health():
return {"ok": True}
return app
@pytest.fixture
def observe_on(monkeypatch):
"""开启观测 + 换一个干净小队列,返回该队列供断言。"""
monkeypatch.setattr(settings, "OBSERVE_ENABLED", True)
monkeypatch.setattr(settings, "OBSERVE_USER", "u")
monkeypatch.setattr(settings, "OBSERVE_PASSWORD", "p")
q = asyncio.Queue(maxsize=100)
monkeypatch.setattr(observe, "_queue", q)
return q
def test_middleware_records_route_template(observe_on):
client = TestClient(_make_probe_app())
r = client.get("/things/42")
assert r.status_code == 200
evt = observe_on.get_nowait()
assert evt["route"] == "/things/{tid}" # 模板,不是 /things/42
assert evt["method"] == "GET"
assert evt["status"] == 200
assert evt["duration_ms"] >= 0
assert evt["service"] and "env" in evt and isinstance(evt["_timestamp"], int)
def test_middleware_skips_health(observe_on):
client = TestClient(_make_probe_app())
client.get("/health")
assert observe_on.empty()
def test_middleware_unmatched_route_is_normalized(observe_on):
client = TestClient(_make_probe_app())
r = client.get("/definitely-not-a-route")
assert r.status_code == 404
evt = observe_on.get_nowait()
assert evt["route"] == "__unmatched__"
assert evt["status"] == 404
def test_middleware_noop_when_disabled(monkeypatch):
monkeypatch.setattr(settings, "OBSERVE_ENABLED", False)
q = asyncio.Queue(maxsize=100)
monkeypatch.setattr(observe, "_queue", q)
client = TestClient(_make_probe_app())
client.get("/things/1")
assert q.empty() # 未配置观测 → 零入队
async def test_collect_batch_drains_up_to_batch_max(monkeypatch):
q = asyncio.Queue(maxsize=100)
monkeypatch.setattr(observe, "_queue", q)
monkeypatch.setattr(settings, "OBSERVE_FLUSH_INTERVAL_SEC", 0.1)
monkeypatch.setattr(settings, "OBSERVE_BATCH_MAX", 200)
for i in range(3):
q.put_nowait({"n": i})
batch = await observe_worker._collect_batch()
assert [e["n"] for e in batch] == [0, 1, 2]
async def test_collect_batch_timeout_returns_empty(monkeypatch):
q = asyncio.Queue(maxsize=100)
monkeypatch.setattr(observe, "_queue", q)
monkeypatch.setattr(settings, "OBSERVE_FLUSH_INTERVAL_SEC", 0.05)
batch = await observe_worker._collect_batch()
assert batch == []
async def test_post_batch_hits_json_ingest_url(monkeypatch):
monkeypatch.setattr(settings, "OBSERVE_ORG", "default")
monkeypatch.setattr(settings, "OBSERVE_STREAM", "app_requests")
captured = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["url"] = str(request.url)
captured["json"] = request.content
return httpx.Response(200, json={"code": 200})
client = httpx.AsyncClient(
base_url="http://oo", transport=httpx.MockTransport(handler)
)
await observe_worker._post_batch(client, [{"route": "/x", "status": 200}])
await client.aclose()
assert captured["url"] == "http://oo/api/default/app_requests/_json"
assert b"/x" in captured["json"]
def test_start_observe_worker_noop_when_not_configured(monkeypatch):
monkeypatch.setattr(settings, "OBSERVE_ENABLED", False)
assert observe_worker.start_observe_worker() is None
async def test_run_loop_survives_post_failure(monkeypatch):
"""_post_batch 抛异常时,loop 不崩溃、继续处理后续批次(best-effort 契约)。"""
q = asyncio.Queue(maxsize=100)
monkeypatch.setattr(observe, "_queue", q)
monkeypatch.setattr(settings, "OBSERVE_FLUSH_INTERVAL_SEC", 0.02)
monkeypatch.setattr(settings, "OBSERVE_BATCH_MAX", 200)
seen: list[list[int]] = []
async def boom(client, batch):
seen.append([e["n"] for e in batch])
raise RuntimeError("boom")
monkeypatch.setattr(observe_worker, "_post_batch", boom)
q.put_nowait({"n": 1})
task = asyncio.create_task(observe_worker._run_loop(None))
try:
for _ in range(50): # 轮询直到第 1 批被处理(失败),最多等 0.5s
await asyncio.sleep(0.01)
if seen:
break
q.put_nowait({"n": 2})
for _ in range(50): # 第 2 批被处理 → 证明失败后 loop 仍存活
await asyncio.sleep(0.01)
if len(seen) >= 2:
break
finally:
task.cancel()
try:
await task
except asyncio.CancelledError:
pass
assert seen == [[1], [2]]
async def test_stop_flushes_remaining_and_closes_client(monkeypatch):
"""stop:cancel 后把剩余事件 best-effort 发出最后一批,并关闭 + 置空 client。"""
q = asyncio.Queue(maxsize=100)
monkeypatch.setattr(observe, "_queue", q)
monkeypatch.setattr(settings, "OBSERVE_ORG", "default")
monkeypatch.setattr(settings, "OBSERVE_STREAM", "app_requests")
monkeypatch.setattr(settings, "OBSERVE_BATCH_MAX", 200)
q.put_nowait({"n": 1})
q.put_nowait({"n": 2})
posted: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
posted["body"] = request.content
return httpx.Response(200, json={"code": 200})
client = httpx.AsyncClient(
base_url="http://oo", transport=httpx.MockTransport(handler)
)
monkeypatch.setattr(observe_worker, "_client", client)
async def _noop() -> None:
return None
task = asyncio.create_task(_noop())
await observe_worker.stop_observe_worker(task)
assert b'"n"' in posted["body"] # 关停时把剩余事件发了出去
assert observe_worker._client is None # client 已关闭并置空
def test_app_has_metrics_middleware():
from app.main import app
names = [m.cls.__name__ for m in app.user_middleware]
assert "RequestMetricsMiddleware" in names
+57
View File
@@ -0,0 +1,57 @@
"""ratelimit 内存桶过期清理(GC)测试。
回归重点:_buckets **全局共享**混着不同窗口(60s 广告 / 3600s 登录 / 86400s 日闸) key
GC 必须按每个 key 自己存的 window_sec判过期,而不是当前调用方的窗口 否则高频的 60s 端点
触发 GC 时会把本该存活更久的 3600s/86400s 计数(如短信日闸)一并删掉,使其被反复清零限流失效
monkeypatch _GC_THRESHOLD 0 强制每次都扫,免造上万条(仿 test_auth 里对 sms._GC_THRESHOLD 的做法)
"""
from __future__ import annotations
from app.core import ratelimit
def test_purge_expired_respects_each_key_own_window(monkeypatch) -> None:
"""短窗口(60s)触发的 GC 只删真正过期的 key,不得删掉仍在自身窗口内的长窗口 key。"""
monkeypatch.setattr(ratelimit, "_GC_THRESHOLD", 0) # 强制每次都扫
ratelimit._buckets.clear()
now = 1_000_000.0
# 日闸:100s 前开窗、window=86400 → 远未过期,必须保留
ratelimit._buckets["sms-send-device-daily:D:IP"] = (now - 100, 7, 86400.0)
# 登录:1800s、window=3600 → 未过期,保留
ratelimit._buckets["sms-login-device:D:IP"] = (now - 1800, 2, 3600.0)
# 广告:120s、window=60 → 已过期,应删
ratelimit._buckets["ad-watch-report:IP2"] = (now - 120, 3, 60.0)
ratelimit._purge_expired(now)
assert "sms-send-device-daily:D:IP" in ratelimit._buckets
assert "sms-login-device:D:IP" in ratelimit._buckets
assert "ad-watch-report:IP2" not in ratelimit._buckets
def test_purge_expired_keeps_long_window_key_older_than_short_window(monkeypatch) -> None:
"""反证旧 bug:日闸 key 已老于 3600s,旧代码在 60s/3600s 端点触发 GC 时会误删它;
现在按自身 86400s 窗口判 未过期 必须保留"""
monkeypatch.setattr(ratelimit, "_GC_THRESHOLD", 0)
ratelimit._buckets.clear()
now = 2_000_000.0
# 3700s 前开窗(> 1 小时),但 window=86400 → 未过期
ratelimit._buckets["sms-send-device-daily:D:IP"] = (now - 3700, 20, 86400.0)
ratelimit._purge_expired(now)
assert "sms-send-device-daily:D:IP" in ratelimit._buckets
def test_purge_expired_noop_below_threshold(monkeypatch) -> None:
"""未超阈值时不扫(即便有过期 key 也不动),避免每次请求都 O(n) 扫全表。"""
monkeypatch.setattr(ratelimit, "_GC_THRESHOLD", 10)
ratelimit._buckets.clear()
now = 3_000_000.0
ratelimit._buckets["stale:IP"] = (now - 999, 1, 60.0) # 早过期,但没超阈值
ratelimit._purge_expired(now)
assert "stale:IP" in ratelimit._buckets # 桶数没超阈值 → 不清理
+232
View File
@@ -0,0 +1,232 @@
"""微信登录 M2 测试:conflict_ticket 令牌、继续绑定(attach/只登入)、换绑(建号+软删+30天限)。
沿用 tests/test_wechat_login.py 风格:HTTP client;微信 codeopenid monkeypatch;
短信走 SMS_MOCK(任意 6 位过)数据变更用"再走一遍 wechat-login 看 openid 落在哪个账号"做行为断言
"""
from __future__ import annotations
import pytest
from app.api.v1 import auth # noqa: F401 (后续测试打桩 verify_and_get_phone 用)
from app.core import security
from app.integrations import wxpay
from app.models.phone_rebind_log import PhoneRebindLog
def _fake_userinfo(openid: str, nickname: str | None = "微信昵称", avatar: str | None = "http://x/a.png"):
def _f(code: str) -> dict:
return {"openid": openid, "nickname": nickname, "avatar_url": avatar, "raw": {}}
return _f
def _sms_occupy(client, phone: str) -> int:
"""用普通短信登录占用一个手机号(register_channel=sms),返回该账号 id。"""
assert client.post("/api/v1/auth/sms/send", json={"phone": phone}).status_code == 200
r = client.post("/api/v1/auth/sms/login", json={"phone": phone, "code": "123456"})
assert r.status_code == 200, r.text
return r.json()["user"]["id"]
def _occupy_via_conflict(client, monkeypatch, openid: str, phone: str, device_id: str) -> dict:
"""微信登录(新 openid)→ 绑同一手机号 → 返回 phone_occupied 的响应体(含 conflict_ticket)。"""
monkeypatch.setattr(wxpay, "code_to_userinfo", _fake_userinfo(openid))
ticket = client.post(
"/api/v1/auth/wechat-login", json={"code": "c", "device_id": device_id}
).json()["bind_ticket"]
r = client.post(
"/api/v1/auth/wechat/bind-phone/sms",
json={"bind_ticket": ticket, "phone": phone, "code": "123456", "device_id": device_id},
)
assert r.status_code == 200, r.text
body = r.json()
assert body["status"] == "phone_occupied"
return body
# ===== Task 1: 模型可导入(建表由 conftest 的 create_all 完成) =====
def test_phone_rebind_log_model_importable() -> None:
assert PhoneRebindLog.__tablename__ == "phone_rebind_log"
# ===== Task 2: conflict_ticket 令牌 =====
def test_conflict_ticket_roundtrip() -> None:
token = security.create_conflict_ticket(
openid="oid1", wechat_nickname="", wechat_avatar_url="http://a", phone="13900139000"
)
claims = security.decode_conflict_ticket(token)
assert claims["openid"] == "oid1"
assert claims["wnk"] == ""
assert claims["wav"] == "http://a"
assert claims["phone"] == "13900139000"
def test_conflict_ticket_wrong_type_rejected() -> None:
# bind_ticket 冒充 conflict_ticket → TokenError(typ 不匹配)
bind = security.create_bind_ticket(openid="oid", wechat_nickname=None, wechat_avatar_url=None)
with pytest.raises(security.TokenError):
security.decode_conflict_ticket(bind)
def test_conflict_ticket_expired_rejected(monkeypatch) -> None:
monkeypatch.setattr(security.settings, "WECHAT_BIND_TICKET_EXPIRE_MINUTES", -1)
token = security.create_conflict_ticket(
openid="oid", wechat_nickname=None, wechat_avatar_url=None, phone="13900139000"
)
with pytest.raises(security.TokenError):
security.decode_conflict_ticket(token)
# ===== Task 3: 占用响应扩展 =====
def test_phone_occupied_returns_conflict_ticket_and_flags(client, monkeypatch) -> None:
phone = "13900139101"
_sms_occupy(client, phone) # 老账号 X(sms,无微信)
body = _occupy_via_conflict(client, monkeypatch, "openid_occ_101", phone, "devO1")
assert body["conflict_ticket"]
assert body["rebind_available"] is True # 首次,未换绑过
assert body["rebind_blocked_days"] == 0
assert body["occupied_account"]["has_wechat"] is False # X 是 sms 账号
# ===== Task 4: 继续绑定 =====
def test_continue_attaches_wechat_and_logs_into_existing(client, monkeypatch) -> None:
"""X 无微信 → 继续绑定并入 openid + 登入 X;之后同 openid 登录直接命中 X。"""
phone = "13900139201"
x_id = _sms_occupy(client, phone) # X:sms 账号,无微信
body = _occupy_via_conflict(client, monkeypatch, "openid_cont_201", phone, "devC1")
r = client.post(
"/api/v1/auth/wechat/conflict/continue",
json={"conflict_ticket": body["conflict_ticket"], "device_id": "devC1"},
)
assert r.status_code == 200, r.text
assert r.json()["status"] == "logged_in"
assert r.json()["token"]["user"]["id"] == x_id # 登入的是老账号 X
# openid 现已并入 X:再走 wechat-login 直接命中 X
r = client.post("/api/v1/auth/wechat-login", json={"code": "c", "device_id": "devC1"})
assert r.json()["status"] == "logged_in"
assert r.json()["token"]["user"]["id"] == x_id
def test_continue_when_existing_has_wechat_logs_in_and_discards_openid(client, monkeypatch) -> None:
"""X 已绑别的微信 → 继续绑定只登入 X、丢弃本次 openid(不覆盖)。"""
phone = "13900139202"
# 先建一个已绑微信 O1 的账号 X(微信登录 O1 + 短信绑号)
monkeypatch.setattr(wxpay, "code_to_userinfo", _fake_userinfo("openid_o1_202"))
t = client.post("/api/v1/auth/wechat-login", json={"code": "c", "device_id": "devC2"}).json()["bind_ticket"]
x = client.post(
"/api/v1/auth/wechat/bind-phone/sms",
json={"bind_ticket": t, "phone": phone, "code": "123456", "device_id": "devC2"},
).json()
x_id = x["token"]["user"]["id"]
# 新 openid O2 撞同号 → 占用(has_wechat=True)→ 继续绑定
body = _occupy_via_conflict(client, monkeypatch, "openid_o2_202", phone, "devC2b")
assert body["occupied_account"]["has_wechat"] is True
r = client.post(
"/api/v1/auth/wechat/conflict/continue",
json={"conflict_ticket": body["conflict_ticket"], "device_id": "devC2b"},
)
assert r.status_code == 200, r.text
assert r.json()["token"]["user"]["id"] == x_id # 登入 X
# O2 被丢弃:再走 wechat-login(O2)→ 仍未命中(need_bind_phone)
monkeypatch.setattr(wxpay, "code_to_userinfo", _fake_userinfo("openid_o2_202"))
assert client.post(
"/api/v1/auth/wechat-login", json={"code": "c", "device_id": "devC2b"}
).json()["status"] == "need_bind_phone"
def test_continue_expired_ticket_returns_401(client, monkeypatch) -> None:
monkeypatch.setattr(security.settings, "WECHAT_BIND_TICKET_EXPIRE_MINUTES", -1)
expired = security.create_conflict_ticket(
openid="oid", wechat_nickname=None, wechat_avatar_url=None, phone="13900139209"
)
r = client.post(
"/api/v1/auth/wechat/conflict/continue",
json={"conflict_ticket": expired, "device_id": "devC3"},
)
assert r.status_code == 401, r.text
# ===== Task 5: 换绑 =====
def test_rebind_creates_new_account_and_binds_openid(client, monkeypatch) -> None:
"""换绑 → 建全新微信账号 Y(≠X)+ openid 落到 Y;老账号 X 被注销(手机号归 Y)。"""
phone = "13900139301"
x_id = _sms_occupy(client, phone)
body = _occupy_via_conflict(client, monkeypatch, "openid_rb_301", phone, "devR1")
r = client.post(
"/api/v1/auth/wechat/conflict/rebind",
json={"conflict_ticket": body["conflict_ticket"], "device_id": "devR1"},
)
assert r.status_code == 200, r.text
y = r.json()["token"]["user"]
assert r.json()["status"] == "logged_in"
assert y["phone"] == phone
assert y["register_channel"] == "wechat"
assert y["id"] != x_id # 是全新账号,不是老账号
# openid 落到 Y:再走 wechat-login 命中 Y
r = client.post("/api/v1/auth/wechat-login", json={"code": "c", "device_id": "devR1"})
assert r.json()["status"] == "logged_in"
assert r.json()["token"]["user"]["id"] == y["id"]
# ===== §10: continue 路径也应用展示身份回填规则 =====
def test_continue_applies_section10(client, monkeypatch) -> None:
"""§10 via M2 attach 路径: X 是默认昵称+null头像的 sms 账号;
continue 绑定微信后,展示昵称/头像应被微信值替换,并体现在响应的 token.user """
phone = "13900139211"
_sms_occupy(client, phone) # 建 X:默认昵称, null avatar
body = _occupy_via_conflict(
client, monkeypatch, "openid_s10", phone, "devS10"
) # fake_userinfo 默认 nickname="微信昵称", avatar="http://x/a.png"
r = client.post(
"/api/v1/auth/wechat/conflict/continue",
json={"conflict_ticket": body["conflict_ticket"], "device_id": "devS10"},
)
assert r.status_code == 200, r.text
user_out = r.json()["token"]["user"]
assert user_out["nickname"] == "微信昵称"
assert user_out["avatar_url"] == "http://x/a.png"
def test_rebind_blocked_within_30_days(client, monkeypatch) -> None:
"""同一手机号 30 天内二次换绑 → 409;占用响应 rebind_available=False。"""
phone = "13900139302"
_sms_occupy(client, phone)
body = _occupy_via_conflict(client, monkeypatch, "openid_rb_302a", phone, "devR2")
assert client.post(
"/api/v1/auth/wechat/conflict/rebind",
json={"conflict_ticket": body["conflict_ticket"], "device_id": "devR2"},
).status_code == 200
# 第二次:新 openid 撞同号 → 占用响应此时 rebind_available=False
body2 = _occupy_via_conflict(client, monkeypatch, "openid_rb_302b", phone, "devR2b")
assert body2["rebind_available"] is False
assert body2["rebind_blocked_days"] >= 1
r = client.post(
"/api/v1/auth/wechat/conflict/rebind",
json={"conflict_ticket": body2["conflict_ticket"], "device_id": "devR2b"},
)
assert r.status_code == 409, r.text
def test_rebind_expired_ticket_returns_401(client, monkeypatch) -> None:
monkeypatch.setattr(security.settings, "WECHAT_BIND_TICKET_EXPIRE_MINUTES", -1)
expired = security.create_conflict_ticket(
openid="oid", wechat_nickname=None, wechat_avatar_url=None, phone="13900139309"
)
r = client.post(
"/api/v1/auth/wechat/conflict/rebind",
json={"conflict_ticket": expired, "device_id": "devR3"},
)
assert r.status_code == 401, r.text
+198
View File
@@ -0,0 +1,198 @@
"""微信登录 M1 测试:bind_ticket 令牌、wechat-login(openid 命中/未命中)、
bind-phone(建号/占用/令牌过期)
沿用 tests/test_auth.py 风格:HTTP client fixture;微信 codeopenid monkeypatch
拦掉(conftest WECHAT_APP_ID/SECRET dummy,不真连微信);短信走 SMS_MOCK(任意 6 位通过)
"""
from __future__ import annotations
import pytest
from app.api.v1 import auth
from app.core import security
from app.integrations import wxpay
def _fake_userinfo(openid: str, nickname: str | None = "微信昵称", avatar: str | None = "http://x/a.png"):
"""返回一个可传给 monkeypatch 的假 code_to_userinfo(忽略 code,固定返回给定 openid)。"""
def _f(code: str) -> dict:
return {"openid": openid, "nickname": nickname, "avatar_url": avatar, "raw": {}}
return _f
# ===== Task 1: bind_ticket 令牌 =====
def test_bind_ticket_roundtrip() -> None:
token = security.create_bind_ticket(openid="oid1", wechat_nickname="", wechat_avatar_url="http://a")
claims = security.decode_bind_ticket(token)
assert claims["openid"] == "oid1"
assert claims["wnk"] == ""
assert claims["wav"] == "http://a"
def test_bind_ticket_wrong_type_rejected() -> None:
# 用 access token 冒充 bind_ticket → TokenError(typ 不匹配)
access, _ = security.create_token(user_id=1, token_type="access")
with pytest.raises(security.TokenError):
security.decode_bind_ticket(access)
def test_bind_ticket_expired_rejected(monkeypatch) -> None:
monkeypatch.setattr(security.settings, "WECHAT_BIND_TICKET_EXPIRE_MINUTES", -1)
token = security.create_bind_ticket(openid="oid", wechat_nickname=None, wechat_avatar_url=None)
with pytest.raises(security.TokenError):
security.decode_bind_ticket(token)
# ===== Task 2: wechat-login =====
def test_wechat_login_new_openid_returns_bind_ticket(client, monkeypatch) -> None:
monkeypatch.setattr(wxpay, "code_to_userinfo", _fake_userinfo("openid_new_1", "小明", "http://x/m.png"))
r = client.post("/api/v1/auth/wechat-login", json={"code": "wxcode1", "device_id": "devA"})
assert r.status_code == 200, r.text
body = r.json()
assert body["status"] == "need_bind_phone"
assert body["bind_ticket"]
assert body["wechat_nickname"] == "小明"
assert body["wechat_avatar_url"] == "http://x/m.png"
assert body["token"] is None
def test_wechat_login_invalid_code_returns_400(client, monkeypatch) -> None:
def _raise(code: str) -> dict:
raise ValueError("微信授权失败: invalid code")
monkeypatch.setattr(wxpay, "code_to_userinfo", _raise)
r = client.post("/api/v1/auth/wechat-login", json={"code": "bad", "device_id": "devA"})
assert r.status_code == 400, r.text
# ===== Task 3: bind-phone/sms =====
def test_wechat_bind_sms_creates_account_then_openid_logs_in(client, monkeypatch) -> None:
"""未占用 → 建微信账号(channel=wechat,昵称头像取微信);再次同 openid 登录 → 直接登入同一账号。"""
monkeypatch.setattr(wxpay, "code_to_userinfo", _fake_userinfo("openid_flow_2", "阿花", "http://x/h.png"))
phone = "13900139002"
# 1) 微信登录 → 未命中 → 拿 ticket
r = client.post("/api/v1/auth/wechat-login", json={"code": "c1", "device_id": "devB"})
ticket = r.json()["bind_ticket"]
assert ticket
# 2) 短信绑号(SMS_MOCK:任意 6 位通过)→ 建号 + 登入
r = client.post(
"/api/v1/auth/wechat/bind-phone/sms",
json={"bind_ticket": ticket, "phone": phone, "code": "123456", "device_id": "devB"},
)
assert r.status_code == 200, r.text
body = r.json()
assert body["status"] == "logged_in"
user = body["token"]["user"]
assert user["phone"] == phone
assert user["register_channel"] == "wechat"
assert user["nickname"] == "阿花"
assert user["avatar_url"] == "http://x/h.png"
uid = user["id"]
# 3) 再次微信登录(同 openid)→ 命中 → 直接登入同一账号
r = client.post("/api/v1/auth/wechat-login", json={"code": "c2", "device_id": "devB"})
assert r.status_code == 200, r.text
body = r.json()
assert body["status"] == "logged_in"
assert body["token"]["user"]["id"] == uid
def test_wechat_bind_sms_phone_occupied(client, monkeypatch) -> None:
"""手机号已被其他账号占用 → 返回 phone_occupied + 原账号信息(不建号)。"""
phone = "13900139003"
# 先用普通短信登录占用该手机号(register_channel=sms)
assert client.post("/api/v1/auth/sms/send", json={"phone": phone}).status_code == 200
r = client.post("/api/v1/auth/sms/login", json={"phone": phone, "code": "123456"})
assert r.status_code == 200, r.text
occupied_nickname = r.json()["user"]["nickname"]
# 微信登录(新 openid)→ 未命中 → ticket
monkeypatch.setattr(wxpay, "code_to_userinfo", _fake_userinfo("openid_occ_3"))
ticket = client.post(
"/api/v1/auth/wechat-login", json={"code": "c", "device_id": "devC"}
).json()["bind_ticket"]
# 绑同一手机号 → 占用
r = client.post(
"/api/v1/auth/wechat/bind-phone/sms",
json={"bind_ticket": ticket, "phone": phone, "code": "123456", "device_id": "devC"},
)
assert r.status_code == 200, r.text
body = r.json()
assert body["status"] == "phone_occupied"
assert body["token"] is None
assert body["occupied_account"]["nickname"] == occupied_nickname
assert body["occupied_account"]["avatar_url"] is None # 短信注册账号无头像 → 序列化为 null
assert body["occupied_account"]["created_at"]
def test_wechat_bind_sms_expired_ticket_returns_401(client, monkeypatch) -> None:
"""过期 bind_ticket → 401。"""
monkeypatch.setattr(security.settings, "WECHAT_BIND_TICKET_EXPIRE_MINUTES", -1)
expired = security.create_bind_ticket(openid="openid_exp", wechat_nickname="x", wechat_avatar_url=None)
r = client.post(
"/api/v1/auth/wechat/bind-phone/sms",
json={"bind_ticket": expired, "phone": "13900139009", "code": "123456", "device_id": "devD"},
)
assert r.status_code == 401, r.text
# ===== Task 4: bind-phone/jverify =====
def test_wechat_bind_jverify_creates_account(client, monkeypatch) -> None:
"""本机号(极光)绑定路径:verify_and_get_phone 拦掉,未占用 → 建号登入。"""
monkeypatch.setattr(wxpay, "code_to_userinfo", _fake_userinfo("openid_jv_5", "极光用户", None))
phone = "13900139005"
# 极光 loginToken→手机号 在 auth 模块命名空间打桩(auth.py 顶部 from ...jiguang import verify_and_get_phone)
monkeypatch.setattr(auth, "verify_and_get_phone", lambda token: phone)
ticket = client.post(
"/api/v1/auth/wechat-login", json={"code": "c", "device_id": "devE"}
).json()["bind_ticket"]
r = client.post(
"/api/v1/auth/wechat/bind-phone/jverify",
json={"bind_ticket": ticket, "login_token": "jgtoken", "device_id": "devE"},
)
assert r.status_code == 200, r.text
body = r.json()
assert body["status"] == "logged_in"
user = body["token"]["user"]
assert user["phone"] == phone
assert user["register_channel"] == "wechat"
assert user["nickname"] == "极光用户"
# 微信 userinfo 隐私脱敏 avatar=None → 头像为空(客户端兜底默认头像)
assert user["avatar_url"] is None
def test_wechat_bind_jverify_expired_ticket_returns_401(client, monkeypatch) -> None:
"""过期 bind_ticket → 401(极光绑号路径,decode 先于极光核验)。"""
monkeypatch.setattr(security.settings, "WECHAT_BIND_TICKET_EXPIRE_MINUTES", -1)
expired = security.create_bind_ticket(openid="openid_jv_exp", wechat_nickname="x", wechat_avatar_url=None)
r = client.post(
"/api/v1/auth/wechat/bind-phone/jverify",
json={"bind_ticket": expired, "login_token": "jgtoken", "device_id": "devE"},
)
assert r.status_code == 401, r.text
def test_wechat_bind_jverify_jiguang_error_returns_502(client, monkeypatch) -> None:
"""极光核验失败(JiguangError)→ 502。"""
monkeypatch.setattr(wxpay, "code_to_userinfo", _fake_userinfo("openid_jv_err"))
def _raise(token: str) -> str:
raise auth.JiguangError("mock jg failure")
monkeypatch.setattr(auth, "verify_and_get_phone", _raise)
ticket = client.post(
"/api/v1/auth/wechat-login", json={"code": "c", "device_id": "devE"}
).json()["bind_ticket"]
r = client.post(
"/api/v1/auth/wechat/bind-phone/jverify",
json={"bind_ticket": ticket, "login_token": "badtoken", "device_id": "devE"},
)
assert r.status_code == 502, r.text
+88
View File
@@ -312,6 +312,94 @@ def test_bind_rejects_openid_already_bound(client, monkeypatch) -> None:
assert r.status_code == 409, r.text
# ===== §10 绑微信时展示身份回填规则 =====
def test_bind_replaces_default_nickname_and_null_avatar(client, monkeypatch) -> None:
"""§10-A: 全默认(昵称=系统默认,头像=null) → 绑微信后两个展示字段都替换为微信值。"""
monkeypatch.setattr(
"app.integrations.wxpay.code_to_userinfo",
lambda code: {"openid": "openid_s10_a", "nickname": "微信昵称A", "avatar_url": "https://x/a.png", "raw": {}},
)
token = _login(client, "13800003001")
r = client.post("/api/v1/wallet/bind-wechat", json={"code": "c"}, headers=_auth(token))
assert r.status_code == 200, r.text
r = client.get("/api/v1/auth/me", headers=_auth(token))
assert r.status_code == 200, r.text
body = r.json()
assert body["nickname"] == "微信昵称A"
assert body["avatar_url"] == "https://x/a.png"
def test_bind_keeps_customized_nickname(client, monkeypatch) -> None:
"""§10-B: 用户已改过昵称 → 绑微信后昵称保留,但 null 头像仍替换为微信头像。"""
monkeypatch.setattr(
"app.integrations.wxpay.code_to_userinfo",
lambda code: {"openid": "openid_s10_b", "nickname": "微信昵称B", "avatar_url": "https://x/b.png", "raw": {}},
)
token = _login(client, "13800003002")
# 先把昵称改成非默认值
r = client.patch(
"/api/v1/user/profile", json={"nickname": "我的名字"}, headers=_auth(token)
)
assert r.status_code == 200, r.text
r = client.post("/api/v1/wallet/bind-wechat", json={"code": "c"}, headers=_auth(token))
assert r.status_code == 200, r.text
r = client.get("/api/v1/auth/me", headers=_auth(token))
body = r.json()
assert body["nickname"] == "我的名字" # 保留自定义昵称
assert body["avatar_url"] == "https://x/b.png" # null 头像被微信头像替换
def test_bind_keeps_customized_avatar(client, monkeypatch) -> None:
"""§10-C: 已有自定义头像 → 绑微信后头像保留,但默认昵称替换为微信昵称。"""
monkeypatch.setattr(
"app.integrations.wxpay.code_to_userinfo",
lambda code: {"openid": "openid_s10_c", "nickname": "微信昵称C", "avatar_url": "https://x/c.png", "raw": {}},
)
token = _login(client, "13800003003")
phone = "13800003003"
# 直接用 DB 给用户写入自定义头像(保持默认昵称)
db = SessionLocal()
try:
user = db.execute(select(User).where(User.phone == phone)).scalar_one()
user.avatar_url = "https://custom/av.png"
db.commit()
finally:
db.close()
r = client.post("/api/v1/wallet/bind-wechat", json={"code": "c"}, headers=_auth(token))
assert r.status_code == 200, r.text
r = client.get("/api/v1/auth/me", headers=_auth(token))
body = r.json()
assert body["nickname"] == "微信昵称C" # 默认昵称被替换
assert body["avatar_url"] == "https://custom/av.png" # 自定义头像保留
def test_bind_wechat_empty_keeps_default(client, monkeypatch) -> None:
"""§10-D: 微信侧 nickname/avatar 均为 None → 不覆盖,展示字段维持原默认值。"""
monkeypatch.setattr(
"app.integrations.wxpay.code_to_userinfo",
lambda code: {"openid": "openid_s10_d", "nickname": None, "avatar_url": None, "raw": {}},
)
token = _login(client, "13800003004")
r = client.get("/api/v1/auth/me", headers=_auth(token))
original_nickname = r.json()["nickname"] # 系统分配的默认昵称
assert original_nickname.startswith("用户")
r = client.post("/api/v1/wallet/bind-wechat", json={"code": "c"}, headers=_auth(token))
assert r.status_code == 200, r.text
r = client.get("/api/v1/auth/me", headers=_auth(token))
body = r.json()
assert body["nickname"] == original_nickname # 默认昵称不变
assert body["avatar_url"] is None # 头像仍为 null
def test_withdraw_reject_refunds(client, monkeypatch) -> None:
"""管理员审核拒绝 → 退回现金 + 单 rejected + 理由写入 fail_reason(用户可见)。"""
monkeypatch.setattr("app.integrations.wxpay.code_to_userinfo", lambda code: {"openid": "openid_reject", "nickname": None, "avatar_url": None, "raw": {}})