Compare commits

...

14 Commits

Author SHA1 Message Date
zhuzihao 4630fd017b feat(ad-revenue): 收益报表按一次比价/领券聚合(trace 合并)+ 微信昵称/签到时间修复 (#103)
- 报表主表改为「单次广告行为」:激励视频=一次观看一行;一次比价/领券=同一 trace_id(整场,
  无 trace 时兜底 ad_session_id)的多条发奖聚成一行,点开看逐条金币复算。中途跳转广告致浮层重弹、
  ad_session_id 变化也能按 trace 归一行。
- 信息流统一 Draw(业务已全切):新增 audit 的 feed_all scene,让「Draw 信息流」筛选覆盖历史误标的
  feed/NULL;聚合行 ad_type 统一 draw;_feed_rows 带出 trace_id 供聚合。
- 父行补 sub_rewards(组内逐条复算明细)、sub_count;row_revenue_yuan=该次发奖广告 eCPM 折算收益之和,
  仅供主表逐行展示、不进合计/趋势(避免与展示侧 total 重复计)。
- 用户 360 概览(AdminUserListItem)返回 wechat_nickname,供收益详情抽屉显示微信昵称。
- 金币记录:签到来自 coin_transaction(存北京 wall-clock),组装时转 UTC 与广告记录统一,修抽屉签到
  时间多 8 小时;签到窗口边界 +8h 对齐;顺带补 _window_conds 引用却从未定义的 _as_utc_naive
  (自定义区间会 NameError)。

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: zzhyyyyy <2685922758@qq.com>
Reviewed-on: #103
Co-authored-by: zhuzihao <zhuzihao@wonderable.ai>
Co-committed-by: zhuzihao <zhuzihao@wonderable.ai>
2026-07-02 08:57:15 +08:00
wuqi 70c2349950 feat(migration): 添加合并 jd_cps_order_fields 和 coupon_session_origin_package 的迁移文件 (#102)
Reviewed-on: #102
Co-authored-by: wuqi <wuqi@wonderable.ai>
Co-committed-by: wuqi <wuqi@wonderable.ai>
2026-07-01 20:21:34 +08:00
zhuzihao 357b2312af chore(deploy): 穿山甲收益拉取 systemd 定时器(每天10:30)+ 运维手册 (#100)
#92 的 scripts/sync_pangle_revenue.py 补 systemd 部署单元,让穿山甲 GroMore T+1 后台收益每天自动拉取入库,免人工敲命令。纯部署/文档文件,不改运行时代码。

- service:oneshot,跑 `--days 3` 回补近 3 天(幂等 upsert,扛偶发漏跑与穿山甲历史订正);10min 硬超时;加固块对齐主服务。
- timer:每天 10:30 触发(穿山甲约 10:00 出数留余量),Persistent 宕机后补跑。
- md:运维手册——上线前置(.env 填 PANGLE_REPORT_* 凭证、子账号需授权否则接口 118)、部署/健康检查/参数/注意事项(join key 用 ad_unit_id 非 code_id 等)。
- 风格对齐现有 deploy/daily-exchange.* 与 deploy/meituan-etl.*。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: zzhyyyyy <2685922758@qq.com>
Reviewed-on: #100
Co-authored-by: zhuzihao <zhuzihao@wonderable.ai>
Co-committed-by: zhuzihao <zhuzihao@wonderable.ai>
2026-06-30 22:59:41 +08:00
zhuzihao 78d970f420 feat: 领券数据看板 — coupon_session 流水表 + admin 聚合接口 (#99)
- 新表 coupon_session(一次领券一行,trace_id 唯一):POST /api/v1/coupon/session
  两段 upsert(发起 started / 收尾 completed·failed·abandoned),记全程耗时、各平台耗时、
  机型/ROM、app_env、trace_url、origin_package(发起来源)。
- admin GET /admin/api/coupon-data:发起/完成数 + 平均耗时 + P5/P50/P95/P99(Python 算分位,
  SQLite 无 percentile)+ 按天/小时趋势 + 明细分页 + join 用户手机号/昵称,app_env 默认 prod;
  另加 GET /admin/api/coupon-data/user-records 供「点手机号看该用户全部领券」抽屉。
- 迁移拆 3 个:建表 coupon_session_table + trace_url 加列 + origin_package 加列
  (建表迁移已被某环境应用后改它不重跑,故新列单独加列迁移)。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: zzhyyyyy <2685922758@qq.com>
Reviewed-on: #99
Co-authored-by: zhuzihao <zhuzihao@wonderable.ai>
Co-committed-by: zhuzihao <zhuzihao@wonderable.ai>
2026-06-30 22:59:36 +08:00
chenshuobo a3a14b484a feat: 接入京东联盟订单到数据大盘 (#90)
改了什么:新增京东联盟订单查询客户端、cps_order 京东字段迁移、CPS 对账接口 platform=all/jd、数据大盘京东订单与佣金聚合。
为什么改:数据大盘需要展示京东 CPS 真实订单佣金,不再保持占位。
验证方式:python -m pytest tests\\test_admin_read.py::test_dashboard_overview tests\\test_cps_admin.py::test_jd_reconcile_updates_dashboard -q;本地真实拉取 2026-06-27 京东订单 3 行,页面显示京东有效 1 单/佣金 ¥0.13。

---------

Co-authored-by: lowmaster-chen <1119780489@qq.com>
Reviewed-on: #90
Co-authored-by: chenshuobo <chenshuobo@wonderable.ai>
Co-committed-by: chenshuobo <chenshuobo@wonderable.ai>
2026-06-30 21:56:58 +08:00
liujiahui 208112ff24 fix(dl): 落地页底图+logo 纳入 git → 修生产 /media 404 毛坯 (#97)
## 背景
用户反馈浏览器落地页仍是「毛坯」(裸黄底、缺优惠券卡/价格卡/logo)。

## 真因(实测线上)
`https://app-api.shaguabijia.com/media/dl.html` 的 **HTML 早已是完善版且已部署**,但它引用的两张静态图被 `.gitignore` 的 `data/media/*` 规则挡在库外、从未随部署进生产 `/media`:

| 资源 | 线上 | 本地/git |
|---|---|---|
| `/media/coupon-page-bg.png`(底图:优惠券卡+跨平台比价价格卡) | **404**(22B `{"detail":"Not Found"}`) | 之前被 gitignore |
| `/media/sb-brand.png`(logo) | **404** | 之前被 gitignore |
| `/media/shaguabijia.apk`(对照) | 200 ✓ | — |

→ 两图加载失败 → 页面回退裸黄底 = 毛坯。

## 改动
照 `dl.html` / `taobao_landing.jpg` 既有白名单写法,把这两张图加进 `.gitignore` 例外并入库。
- `.gitignore`:`!data/media/coupon-page-bg.png`、`!data/media/sb-brand.png`
- 入库两图,md5 与原型一致(coupon-page-bg `f2adf28a` / sb-brand `e5b6e004`)

**无代码/HTML 改动**(dl.html 本就对齐原型,见上一 commit `6d4cbc8`)。

## 部署后生效
合并并部署本分支后,生产 `/media/` 即含这两图,落地页恢复完整。对应 6-29 UI 对齐 #10。

---------

Co-authored-by: no_gen_mu <liujianhishen@gmail.com>
Reviewed-on: #97
Co-authored-by: liujiahui <liujiahui@wonderable.ai>
Co-committed-by: liujiahui <liujiahui@wonderable.ai>
2026-06-30 20:48:31 +08:00
zhuzihao 32f300b5a2 调现金接口支持目标账户(金币现金 / 邀请奖励金)+ overview 带邀请余额 (#95)
- GrantCashRequest 加 account(coin_cash / invite_cash,默认 coin_cash 兼容旧调用)
- 调现金接口按 account 分支:邀请账户走 grant_invite_cash,余额读取/扣负保护/审计
  detail 都按对应账户走;两本账物理隔离不可串
- 用户 360 overview 返回 invite_cash_balance_cents(admin 调现金弹窗展示邀请余额)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: zzhyyyyy <2685922758@qq.com>
Reviewed-on: #95
Co-authored-by: zhuzihao <zhuzihao@wonderable.ai>
Co-committed-by: zhuzihao <zhuzihao@wonderable.ai>
2026-06-30 12:02:56 +08:00
zhuzihao 81639ed388 fix(ad-revenue): 报表收益按满额钳顶 + 新增 app_env 过滤 (#93)
广告收益报表两处隐患修复(admin/repositories/ad_revenue.py + routers/ad_revenue.py):

- 收益钳顶:单次展示收益原用裸 parse_ecpm_yuan、未钳上限;发奖侧 calculate_ad_reward_coin
  已钳 AD_ECPM_MAX_FEN(¥500 CPM)。异常/伪造天价 eCPM 会让报表预估收益虚高任意大。改为
  min(parse_ecpm_yuan, AD_ECPM_MAX_FEN/100)/1000,与发奖同口径。
- app_env 过滤:报表聚合原无 app_env 过滤,测试应用假 eCPM(如 ¥678 CPM)会进正式收益合计/平均。
  新增 app_env 参数(repository + router Query),显式传 prod/test 才过滤;默认仍全部(不擅自改成
  默认排除 test:本地 dev 库多为 test 会空、且属产品口径)。穿山甲后台收益列暂未联动此过滤。

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: zzhyyyyy <2685922758@qq.com>
Reviewed-on: #93
Co-authored-by: zhuzihao <zhuzihao@wonderable.ai>
Co-committed-by: zhuzihao <zhuzihao@wonderable.ai>
2026-06-29 23:11:41 +08:00
zhuzihao 5efe624340 admin 反馈/低价审核接口增强 + 反馈采集端环境(版本/机型) (#94)
- 反馈:新增 GET /admin/api/feedbacks/summary(待审核/已采纳/未采纳/合计计数);
  列表联表带出完整手机号 + 昵称(供 admin 点手机号查该用户全部反馈)
- 低价审核:列表加 sort_by/sort_order(提交时间排序);联表带出手机号/昵称,
  并按 comparison_record_id 关联比价记录带出 trace_id/trace_url + 机型/ROM/Android/app 版本
- feedback 表新增 app_version/device_model/rom_name/android_version 四列 + alembic 迁移;
  /api/v1/feedback 提交接口接收并落库,供 admin 反馈页展示「提交版本号」「机型OS版本」
  (字段可空,旧端不传不影响提交)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: zzhyyyyy <2685922758@qq.com>
Reviewed-on: #94
Co-authored-by: zhuzihao <zhuzihao@wonderable.ai>
Co-committed-by: zhuzihao <zhuzihao@wonderable.ai>
2026-06-29 23:11:30 +08:00
zhuzihao b622f76a02 feat(ad-revenue): 接入穿山甲 GroMore 数据 API 拉取后台收益(预估+收益API) (#92)
- 新增穿山甲 GroMore「聚合数据报告 API」对接,按天 T+1 拉取后台 revenue(预估)与
  api_revenue(收益API,更接近结算),在广告收益报表大盘 + 按天趋势级展示,与客户端自报
  eCPM 折算的预估并列对照;逐条广告事件行不动(仍是客户端预估)。
- 链路:integrations/pangle_report.py(MD5 签名,与官方文档两个测试向量逐字节一致)→
  scripts/sync_pangle_revenue.py(拉昨天/--days 回补)→ 新表 ad_pangle_daily_revenue
  (repositories/ad_pangle_revenue.py:upsert + 按日聚合)→ admin/repositories/ad_revenue.py
  汇总,新增 total_pangle_revenue_yuan / total_pangle_api_revenue_yuan / daily[].pangle_*。
- 穿山甲无用户/类型/场景维度:仅全量视图(未按 user/ad_type/feed_scene 过滤)给值,否则置
  None;join key 用 ad_unit_id(=客户端配的 104xxx),非 code_id。
- 新增配置 PANGLE_REPORT_USER_ID/ROLE_ID/SECURITY_KEY(≠发奖 m-key)+ site_id→应用映射;
  含单测(签名向量+分页解析);已真连穿山甲验证。

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: zzhyyyyy <2685922758@qq.com>
Reviewed-on: #92
Co-authored-by: zhuzihao <zhuzihao@wonderable.ai>
Co-committed-by: zhuzihao <zhuzihao@wonderable.ai>
2026-06-29 09:54:13 +08:00
wuqi c0b67fd879 feat(launch_confirm): 新增启动确认样本列表内部接口(供 pricebot distill 回读) (#91)
补齐「启动确认窗自进化闭环」里 app-server 这一侧的读路径:pricebot 的 launch_confirm
LLM 兜底成功样本会 server→server 落到 app-server 的 launch_confirm_sample 表;本提交
新增「样本列表查询」内部接口, 供 pricebot 的 scripts/distill_launch_confirm.py 回读这些
样本、按「宿主包 × 文案变体」聚合沉淀出候选规则。

- app/api/internal/launch_confirm.py: 新增样本列表内部接口(server→server, 不走用户 JWT)
- app/repositories/launch_confirm_sample.py: 样本列表查询
- app/schemas/launch_confirm_sample.py: 列表响应 schema

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Reviewed-on: #91
Co-authored-by: wuqi <wuqi@wonderable.ai>
Co-committed-by: wuqi <wuqi@wonderable.ai>
2026-06-28 21:40:42 +08:00
xiebing 2ed62c789f feat(h5): 我的页改 H5(mine 页 + shared bridge/api) (#89)
- h5/mine/index.html: 个人中心「我的页」H5 实现
- h5/shared/bridge.js, api.js: H5↔客户端桥与 API 封装(mine 依赖)

Reviewed-on: #89
Co-authored-by: xiebing <xiebing@wonderable.ai>
Co-committed-by: xiebing <xiebing@wonderable.ai>
2026-06-28 14:16:32 +08:00
chenshuobo 1548406f29 美团 feed 改用离线库 + 三 tab 降级兜底(替换 main 的实时版 feed) (#88)
Co-authored-by: lowmaster-chen <1119780489@qq.com>
Co-authored-by: chenshuobo <1119780489@qq.com>
Reviewed-on: #88
Co-authored-by: chenshuobo <chenshuobo@wonderable.ai>
Co-committed-by: chenshuobo <chenshuobo@wonderable.ai>
2026-06-28 10:38:31 +08:00
guke b7b958ed58 perf(pricebot): 透传复用共享 httpx 单例,免每请求重建 SSL 上下文 + 绕过进程代理 (#87)
coupon/compare 透传原先每请求 async with httpx.AsyncClient(...) 新建:
每次重建 SSL 上下文(加载 certifi CA)实测 ~1s+,而 pricebot 是纯 HTTP
透传根本用不到 TLS;且 trust_env 默认 True 会把 http://localhost:8000
经进程代理(Clash)再绕几秒。

抽 app/core/pricebot_client.py 共享单例:trust_env=False 直连,lifespan
启动预热(SSL 一次性成本付在启动)、关停 aclose;timeout 下放到 .post()
保留 coupon 30s / compare 60s 差异。keep-alive 复用 TCP,每帧降到个位数 ms。

---------

Co-authored-by: guke <guke@autohome.com.cn>
Reviewed-on: #87
Co-authored-by: guke <guke@wonderable.ai>
Co-committed-by: guke <guke@wonderable.ai>
2026-06-28 09:36:31 +08:00
74 changed files with 11464 additions and 137 deletions
+20
View File
@@ -58,6 +58,13 @@ MT_CPS_DEFAULT_SID=sgbjia
# 线上国内服务器留空(=直连)。留空且本机直连失败时 /feed、/coupons、/top-sales 会返回空。
MT_CPS_PROXY=
# ===== 京东联盟 CPS =====
# 京东联盟/京东宙斯开放平台创建应用后填写。AUTH_KEY 是工具商授权 key,自有应用可留空。
JD_UNION_APP_KEY=
JD_UNION_APP_SECRET=
JD_UNION_SITE_ID=
JD_UNION_AUTH_KEY=
# ===== Pricebot 上游 (领券/比价业务透传目标) =====
# 客户端调本服务的 /api/v1/coupon/step 等,我们透传到 pricebot-backend。
# 本地开发用 localhost:8000。生产部署改成内网地址(如 http://pricebot.internal:8000)。
@@ -117,3 +124,16 @@ PANGLE_REWARD_SECRET=
# ⚠️ 仅本地联调:true 时开放 POST /api/v1/ad/test-grant,让 debug 客户端看完广告直接发奖,
# 验证"看广告→金币到账"全链路(未部署公网、穿山甲 S2S 打不到本地时用)。生产必须 false(绕过反作弊)。
AD_REWARD_TEST_GRANT_ENABLED=false
# ===== 穿山甲 GroMore 数据 API(按天拉收益报表,供后台广告收益报表的「穿山甲后台收益」)=====
# ⚠️ 与上面发奖回调的 m-key 是【两套不同凭证】:这三样在穿山甲后台「接入中心 → GroMore-API →
# 聚合数据报告 API」文档页领取。只读拉取 GroMore 天级 revenue(预估)/ api_revenue(收益Api),
# 不参与发奖。三样齐全才生效;留空 = scripts/sync_pangle_revenue 直接 no-op。
# 子账号(role_id≠user_id)需主账号在「角色管理」授予「查看全部数据」权限,否则查不到收益(接口 118);
# role_id 填成 = user_id 即查主账号数据。同步:线上每天 ~10:30 由 timer 跑 python -m scripts.sync_pangle_revenue。
PANGLE_REPORT_USER_ID=0
PANGLE_REPORT_ROLE_ID=0
PANGLE_REPORT_SECURITY_KEY=
# GroMore AppId(报表 site_id 维度)→ 应用环境;默认取现网两个应用,按需覆盖。
PANGLE_REPORT_SITE_ID_PROD=5830519
PANGLE_REPORT_SITE_ID_TEST=5832303
+5 -2
View File
@@ -23,14 +23,17 @@ dist/
*.db-journal
*.db-shm
*.db-wal
# data/ 整体不入库(运行时数据/上传文件/大二进制)。例外:邀请落地页 dl.html——
# 它既是生产落地页又是本地测试资产,纳入 git 便于同事一致测试
# data/ 整体不入库(运行时数据/上传文件/大二进制)。例外:邀请落地页 dl.html 及其
# 引用的静态插画(coupon-page-bg.png 底图 + sb-brand.png logo)——既是生产落地页资产
# 又是本地测试资产,纳入 git 便于同事一致测试、且随部署进生产 /media(否则线上 404→落地页毛坯)。
# (见 docs/邀请功能-实现原理与本地测试.md)。其余(avatars/ / *.apk / app.db 等)仍忽略。
data/*
!data/media/
data/media/*
!data/media/dl.html
!data/media/taobao_landing.jpg
!data/media/coupon-page-bg.png
!data/media/sb-brand.png
secrets/*
!secrets/.gitkeep
@@ -0,0 +1,26 @@
"""merge jd_cps_order_fields and coupon_session_origin_package heads
Revision ID: 761ef181ce7c
Revises: coupon_session_origin_package, jd_cps_order_fields
Create Date: 2026-07-01 13:52:16.068808
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = '761ef181ce7c'
down_revision: Union[str, Sequence[str], None] = ('coupon_session_origin_package', 'jd_cps_order_fields')
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
pass
def downgrade() -> None:
pass
@@ -0,0 +1,63 @@
"""ad_pangle_daily_revenue: 穿山甲 GroMore 天级收益报表(后台结算口径)
新建表存放从 GroMore 数据 API 按天拉取的收益(revenue 预估 + api_revenue 收益Api),
粒度 = 日期 × 应用(app_env) × 代码位(our_code_id) × 广告源(adn)。供广告收益报表的
汇总/趋势级展示「穿山甲后台收益」,与客户端自报 eCPM 折算的预估互为对照。
Revision ID: ad_pangle_daily_revenue
Revises: 7db22acee504
Create Date: 2026-06-28
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "ad_pangle_daily_revenue"
down_revision = "7db22acee504"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.create_table(
"ad_pangle_daily_revenue",
sa.Column("id", sa.Integer(), primary_key=True, autoincrement=True),
sa.Column("report_date", sa.String(length=10), nullable=False),
sa.Column("app_env", sa.String(length=16), nullable=False),
sa.Column("site_id", sa.String(length=32), nullable=True),
sa.Column("our_code_id", sa.String(length=64), nullable=False),
sa.Column("adn", sa.String(length=16), nullable=False, server_default=""),
sa.Column("revenue_yuan", sa.Float(), nullable=False, server_default="0"),
sa.Column("api_revenue_yuan", sa.Float(), nullable=True),
sa.Column("ecpm", sa.String(length=32), nullable=True),
sa.Column("impressions", sa.Integer(), nullable=False, server_default="0"),
sa.Column("currency", sa.String(length=8), nullable=False, server_default="cny"),
sa.Column(
"synced_at",
sa.DateTime(timezone=True),
server_default=sa.func.now(),
nullable=False,
),
sa.UniqueConstraint(
"report_date", "app_env", "our_code_id", "adn", name="uq_ad_pangle_daily"
),
)
op.create_index(
"ix_ad_pangle_daily_revenue_report_date",
"ad_pangle_daily_revenue",
["report_date"],
)
op.create_index(
"ix_ad_pangle_daily_revenue_our_code_id",
"ad_pangle_daily_revenue",
["our_code_id"],
)
def downgrade() -> None:
op.drop_index("ix_ad_pangle_daily_revenue_our_code_id", table_name="ad_pangle_daily_revenue")
op.drop_index("ix_ad_pangle_daily_revenue_report_date", table_name="ad_pangle_daily_revenue")
op.drop_table("ad_pangle_daily_revenue")
@@ -0,0 +1,32 @@
"""coupon_session 加 origin_package 列(发起来源 App 包名 → admin「发起平台」)
null=App 内(傻瓜比价首页)发起,非空=从美团/淘宝/京东弹券发起。与 trace_url 同理单独成迁移,
已建表环境靠它补列、全新环境顺序应用,不重复加列。
Revision ID: coupon_session_origin_package
Revises: coupon_session_trace_url
Create Date: 2026-06-30 00:00:00.000000
"""
from collections.abc import Sequence
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "coupon_session_origin_package"
down_revision: str | Sequence[str] | None = "coupon_session_trace_url"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def upgrade() -> None:
with op.batch_alter_table("coupon_session", schema=None) as batch_op:
batch_op.add_column(sa.Column("origin_package", sa.String(length=64), nullable=True))
def downgrade() -> None:
with op.batch_alter_table("coupon_session", schema=None) as batch_op:
batch_op.drop_column("origin_package")
+81
View File
@@ -0,0 +1,81 @@
"""coupon session table(领券任务全程流水 → admin「领券数据」看板数据源)
一次领券一行(trace_id 唯一):客户端 POST /api/v1/coupon/session 两段上报 —— 发起建行
(status=started)、收尾(completed/failed/abandoned)按 trace_id 更新同一行。记全程耗时
elapsed_ms + 各平台耗时 platform_elapsed + 机型/ROM,供 admin 算发起/完成数、耗时分位、机型维度。
Revision ID: coupon_session_table
Revises: feedback_submit_env
Create Date: 2026-06-30 00:00:00.000000
"""
from collections.abc import Sequence
import sqlalchemy as sa
from sqlalchemy.dialects import postgresql
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "coupon_session_table"
down_revision: str | Sequence[str] | None = "feedback_submit_env"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
# PG 用 JSONB,SQLite(本地/测试)退化为通用 JSON(同 model 的 _JSON variant)。
_JSON = sa.JSON().with_variant(postgresql.JSONB(), "postgresql")
def upgrade() -> None:
op.create_table(
"coupon_session",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("trace_id", sa.String(length=64), nullable=False),
sa.Column("device_id", sa.String(length=64), nullable=False),
sa.Column("user_id", sa.Integer(), nullable=True),
sa.Column("status", sa.String(length=16), nullable=False),
sa.Column("app_env", sa.String(length=16), nullable=True),
sa.Column("platforms", _JSON, nullable=True),
sa.Column("device_model", sa.String(length=128), nullable=True),
sa.Column("rom", sa.String(length=64), nullable=True),
sa.Column("started_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("started_date", sa.Date(), nullable=False),
sa.Column("finished_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("elapsed_ms", sa.Integer(), nullable=True),
sa.Column("platform_elapsed", _JSON, nullable=True),
sa.Column("claimed_count", sa.Integer(), nullable=True),
sa.Column(
"created_at",
sa.DateTime(timezone=True),
server_default=sa.text("(CURRENT_TIMESTAMP)"),
nullable=False,
),
sa.Column(
"updated_at",
sa.DateTime(timezone=True),
server_default=sa.text("(CURRENT_TIMESTAMP)"),
nullable=False,
),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("trace_id", name="uq_coupon_session_trace"),
)
with op.batch_alter_table("coupon_session", schema=None) as batch_op:
batch_op.create_index(
batch_op.f("ix_coupon_session_user_id"), ["user_id"], unique=False
)
batch_op.create_index(
batch_op.f("ix_coupon_session_app_env"), ["app_env"], unique=False
)
# admin 主聚合/筛选:按上海自然日 + 环境。
batch_op.create_index(
"ix_coupon_session_date_env", ["started_date", "app_env"], unique=False
)
def downgrade() -> None:
with op.batch_alter_table("coupon_session", schema=None) as batch_op:
batch_op.drop_index("ix_coupon_session_date_env")
batch_op.drop_index(batch_op.f("ix_coupon_session_app_env"))
batch_op.drop_index(batch_op.f("ix_coupon_session_user_id"))
op.drop_table("coupon_session")
@@ -0,0 +1,32 @@
"""coupon_session 加 trace_url 列(pricebot done 帧公网调试链接)
建表迁移 coupon_session_table 落地后才追加本列,故单独成一个迁移:已建表的环境(本地/已 upgrade 过)
靠它补列,全新环境则「建表(无 trace_url)→ 本迁移加列」,两条路一致、不重复加列。
Revision ID: coupon_session_trace_url
Revises: coupon_session_table
Create Date: 2026-06-30 00:00:00.000000
"""
from collections.abc import Sequence
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision: str = "coupon_session_trace_url"
down_revision: str | Sequence[str] | None = "coupon_session_table"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None
def upgrade() -> None:
with op.batch_alter_table("coupon_session", schema=None) as batch_op:
batch_op.add_column(sa.Column("trace_url", sa.String(length=512), nullable=True))
def downgrade() -> None:
with op.batch_alter_table("coupon_session", schema=None) as batch_op:
batch_op.drop_column("trace_url")
+34
View File
@@ -0,0 +1,34 @@
"""feedback 加提交端环境快照(app_version / device_model / rom_name / android_version)
Revision ID: feedback_submit_env
Revises: ad_pangle_daily_revenue
Create Date: 2026-06-29 00:00:00.000000
admin 用户反馈页要展示「提交版本号」「机型OS版本」,需在提交时落库端环境。仅新增可空列,
SQLite 原生支持 add_column、不用 batch;downgrade 的 drop_column 在 SQLite 走 batch 兜底。
历史反馈无此快照 → 留 NULL(无法回填);客户端改版带上后的新反馈才有值。
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "feedback_submit_env"
down_revision: Union[str, Sequence[str], None] = "ad_pangle_daily_revenue"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column("feedback", sa.Column("app_version", sa.String(length=32), nullable=True))
op.add_column("feedback", sa.Column("device_model", sa.String(length=64), nullable=True))
op.add_column("feedback", sa.Column("rom_name", sa.String(length=32), nullable=True))
op.add_column("feedback", sa.Column("android_version", sa.String(length=16), nullable=True))
def downgrade() -> None:
with op.batch_alter_table("feedback") as batch_op:
batch_op.drop_column("android_version")
batch_op.drop_column("rom_name")
batch_op.drop_column("device_model")
batch_op.drop_column("app_version")
+56
View File
@@ -0,0 +1,56 @@
"""add jd cps order fields
Revision ID: jd_cps_order_fields
Revises: 7db22acee504
Create Date: 2026-06-28 20:30:00.000000
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "jd_cps_order_fields"
down_revision = "7db22acee504"
branch_labels = None
depends_on = None
def upgrade() -> None:
with op.batch_alter_table("cps_order") as batch_op:
batch_op.add_column(
sa.Column("platform", sa.String(length=20), nullable=False, server_default="meituan")
)
batch_op.add_column(sa.Column("external_order_id", sa.String(length=128), nullable=True))
batch_op.add_column(sa.Column("external_row_id", sa.String(length=128), nullable=True))
batch_op.add_column(sa.Column("estimated_commission_cents", sa.Integer(), nullable=True))
batch_op.add_column(sa.Column("actual_commission_cents", sa.Integer(), nullable=True))
batch_op.add_column(sa.Column("jd_valid_code", sa.String(length=16), nullable=True))
batch_op.add_column(sa.Column("settle_month", sa.String(length=16), nullable=True))
batch_op.add_column(sa.Column("site_id", sa.String(length=128), nullable=True))
batch_op.add_column(sa.Column("position_id", sa.String(length=128), nullable=True))
batch_op.add_column(sa.Column("pid", sa.String(length=128), nullable=True))
batch_op.add_column(sa.Column("sub_union_id", sa.String(length=128), nullable=True))
batch_op.create_index("ix_cps_order_platform", ["platform"])
batch_op.create_index("ix_cps_order_external_order_id", ["external_order_id"])
batch_op.create_index("ix_cps_order_external_row_id", ["external_row_id"])
batch_op.create_index("ix_cps_order_jd_valid_code", ["jd_valid_code"])
def downgrade() -> None:
with op.batch_alter_table("cps_order") as batch_op:
batch_op.drop_index("ix_cps_order_jd_valid_code")
batch_op.drop_index("ix_cps_order_external_row_id")
batch_op.drop_index("ix_cps_order_external_order_id")
batch_op.drop_index("ix_cps_order_platform")
batch_op.drop_column("sub_union_id")
batch_op.drop_column("pid")
batch_op.drop_column("position_id")
batch_op.drop_column("site_id")
batch_op.drop_column("settle_month")
batch_op.drop_column("jd_valid_code")
batch_op.drop_column("actual_commission_cents")
batch_op.drop_column("estimated_commission_cents")
batch_op.drop_column("external_row_id")
batch_op.drop_column("external_order_id")
batch_op.drop_column("platform")
+2
View File
@@ -21,6 +21,7 @@ from app.admin.routers.audit import router as audit_router
from app.admin.routers.auth import router as auth_router
from app.admin.routers.comparison import router as comparison_router
from app.admin.routers.config import router as config_router
from app.admin.routers.coupon_data import router as coupon_data_router
from app.admin.routers.cps import router as cps_router
from app.admin.routers.dashboard import router as dashboard_router
from app.admin.routers.device_liveness import router as device_liveness_router
@@ -101,6 +102,7 @@ admin_app.include_router(audit_router)
admin_app.include_router(config_router)
admin_app.include_router(comparison_router)
admin_app.include_router(cps_router)
admin_app.include_router(coupon_data_router)
admin_app.include_router(ad_audit_router)
admin_app.include_router(ad_config_router)
admin_app.include_router(ad_revenue_router)
+7 -1
View File
@@ -136,12 +136,16 @@ def _feed_scene_matches(rec: AdFeedRewardRecord, scene: str | None) -> bool:
"""该信息流记录是否落入请求的展示筛选 scene。
- scene=="feed":ad_type in ("feed", NULL)(旧数据 NULL 视为 feed,向后兼容)
- scene=="draw":ad_type=="draw"
- scene=="feed_all":所有信息流(feed/draw/NULL 都要)——业务已全切 Draw 信息流,收益报表把「Draw 信息流」
当作整个信息流口径(含历史误标 feed/NULL),用它避免筛选漏历史。
- scene 为 None:不筛(两类都要)。
"""
if scene == "feed":
return rec.ad_type in (None, "feed")
if scene == "draw":
return rec.ad_type == "draw"
if scene == "feed_all":
return True
return True
@@ -184,6 +188,7 @@ def _feed_rows(
"record_id": rec.id,
"user_id": rec.user_id,
"ad_session_id": rec.ad_session_id,
"trace_id": rec.trace_id,
"app_env": rec.app_env,
"our_code_id": rec.our_code_id,
"created_at": rec.created_at,
@@ -209,6 +214,7 @@ def _feed_rows(
"record_id": rec.id,
"user_id": rec.user_id,
"ad_session_id": rec.ad_session_id,
"trace_id": rec.trace_id,
"app_env": rec.app_env,
"our_code_id": rec.our_code_id,
"created_at": rec.created_at,
@@ -241,7 +247,7 @@ def audit_rows(
rows: list[dict] = []
if scene in (None, "reward_video"):
rows.extend(_reward_video_rows(db, date=date, user_id=user_id))
if scene in (None, "feed", "draw"):
if scene in (None, "feed", "draw", "feed_all"):
rows.extend(_feed_rows(db, date=date, user_id=user_id, scene=scene))
return rows
+144 -37
View File
@@ -3,9 +3,11 @@
只读。每行 = 一次广告事件(不再按用户聚合):
- **激励视频**:一次观看 = 1 条展示(ad_ecpm)+ 1 条发奖(ad_reward),按 ad_session_id 合并成一行,
直接给出 eCPM / 收益 + 状态 / 应发 / 实发 / 一致;点开看该条金币复算因子。
- **信息流**:轮播每条展示各一行(impressionId 各自独立);整场发奖(ad_feed_reward,client_event_id)
与逐条展示无法对应,单独成「纯发奖」行。
- 兜底:有展示无发奖(中途关 / 未达发奖)、有发奖无展示(未上报 eCPM)都各自成行。
- **信息流(比价/领券)**:一次比价 / 一次领券 = 一条整场发奖(ad_feed_reward)一行,给出 eCPM /
发奖金币 + 应发 / 实发 / 一致;点开看金币复算因子。⚠️ draw 的逐条展示(ad_ecpm,impressionId 各自
独立、与整场发奖无公共键、无法归到「哪一次」)**不再单独占行**(2026-07 按「一次比价/领券放一块」调整)——
其展示数 / eCPM / 预估收益仍进全量统计(合计 / 趋势 / 分类大盘 / 穿山甲对照),只是主表不逐条铺开。
- 兜底:激励视频有展示无发奖(中途关 / 未达发奖)、有发奖无展示(未上报 eCPM)仍各自成行。
展示与收益来自 ad_ecpm_record(收益 = eCPM元 ÷ 1000);应发 / 实发金币复用金币审计逐条复算
(ad_audit.audit_rows,与正式发奖同一公式口径,不另写公式)。合计与对账在全量上统计,
@@ -31,6 +33,7 @@ from app.admin.repositories import stats as admin_stats
from app.core import rewards
from app.models.ad_ecpm import AdEcpmRecord
from app.models.user import User
from app.repositories import ad_pangle_revenue
def _cn_hour(dt: datetime) -> int:
@@ -57,14 +60,6 @@ def _date_range(date_from: str, date_to: str) -> list[str]:
_AUDIT_SCENES = {"reward_video", "feed", "draw"}
def _event_ad_type(row: dict) -> str:
"""纯发奖事件行的 ad_type:信息流行用 audit 带回的真实 ad_type(feed/draw),回退 feed;
激励视频行恒 reward_video。不再用 scene 硬映射,避免把 draw 丢成 feed。"""
if row["scene"] == "reward_video":
return "reward_video"
return row.get("ad_type") or "feed"
# 发奖复算明细字段(展开下钻看「金币怎么算出来的」)——从 audit 行原样取这些 key。
_REWARD_DETAIL_KEYS = (
"record_id", "created_at", "status", "ecpm", "ecpm_factor", "units",
@@ -86,6 +81,7 @@ def ad_revenue_report(
user_id: int | None = None,
ad_type: str | None = None,
feed_scene: str | None = None,
app_env: str | None = None,
granularity: str = "day",
limit: int = 500,
offset: int = 0,
@@ -106,9 +102,14 @@ def ad_revenue_report(
# 同时保留全量列表,未被展示合并的成「纯发奖」事件。
reward_by_session: dict[tuple[int, str], list[dict]] = {}
all_reward_rows: list[dict] = []
# 报表 ad_type 直接当 audit scene 用(取值一致);未知/无效 ad_type 不取发奖行。draw 在此被
# 正确传成 scene="draw",audit 会按 ad_type 筛出 Draw 发奖,不再丢成 feed
audit_scene = ad_type if ad_type in _AUDIT_SCENES else None
# 报表 ad_type audit scene:reward_video/feed 直传;**draw(前端「Draw 信息流」)映射成 feed_all**
# ——业务已全切 Draw,把「Draw 信息流」当作整个信息流口径(含历史误标 feed/NULL),否则筛选会漏历史
if ad_type == "draw":
audit_scene = "feed_all"
elif ad_type in _AUDIT_SCENES:
audit_scene = ad_type
else:
audit_scene = None
if ad_type is None or audit_scene is not None:
for d in _date_range(date_from, date_to):
for row in ad_audit.audit_rows(db, date=d, user_id=user_id, scene=audit_scene):
@@ -138,7 +139,10 @@ def ad_revenue_report(
)
if user_id is not None:
stmt = stmt.where(AdEcpmRecord.user_id == user_id)
if ad_type is not None:
if ad_type == "draw":
# draw = 所有信息流展示(业务已全 Draw,含历史误标 feed);展示行只进统计,不占主表行
stmt = stmt.where(AdEcpmRecord.ad_type.in_(["draw", "feed"]))
elif ad_type is not None:
stmt = stmt.where(AdEcpmRecord.ad_type == ad_type)
for rec in db.execute(stmt).scalars():
rwd = _pop_reward(rec.user_id, rec.ad_session_id)
@@ -155,10 +159,16 @@ def ad_revenue_report(
"has_impression": True,
"impressions": 1,
"ecpm": rec.ecpm_raw,
# 单次展示收益(元)= eCPM元 ÷ 1000(每千次→单次);与发奖同源解析,口径一致。
"revenue_yuan": round(rewards.parse_ecpm_yuan(rec.ecpm_raw) / 1000.0, 6),
# 单次展示收益(元)= eCPM元 ÷ 1000(每千次→单次)。eCPM 先钳到 AD_ECPM_MAX_FEN(¥500 CPM)
# 再折收益,与发奖口径 [rewards.calculate_ad_reward_coin] 一致(2026-06-29 修:原裸 parse_ecpm_yuan
# 不钳,伪造/异常天价 eCPM 会把报表预估收益冲到任意大;金币侧已钳、收益侧漏钳)。
"revenue_yuan": round(
min(rewards.parse_ecpm_yuan(rec.ecpm_raw), rewards.AD_ECPM_MAX_FEN / 100.0) / 1000.0, 6,
),
"adn": rec.adn,
"slot_id": rec.slot_id,
"sub_rewards": [],
"sub_count": 1,
}
if rwd is not None:
ev.update({
@@ -178,33 +188,88 @@ def ad_revenue_report(
})
events.append(ev)
# 3) 未被展示合并的发奖行 → 「纯发奖」事件(信息流整场发奖 / 有发奖无展示)。
# 收益恒 0(收益只算展示侧,避免与展示行重复计)。
# 3) 未被展示合并的发奖行 → 事件:
# - 激励视频(reward_video):逐条成「纯发奖」事件(每次一个 ad_session_id;有发奖无展示等)。
# - 信息流(feed/draw):同一次比价/领券的多条广告共享**整场 ad_session_id**(客户端整场复用),
# 按 (user_id, ad_session_id) 聚成**一次比价 / 一次领券**父事件;sub_rewards 为组内逐条明细,
# 应发/实发取组内合计;业务已全 Draw → 类型统一 "draw"。session 缺失(极少旧数据)各自单独成组。
feed_groups: dict[tuple[int, str], list[dict]] = {}
for row in all_reward_rows:
if row["record_id"] in used_reward_ids:
continue
if row["scene"] == "reward_video":
events.append({
"event_key": f"rwd-{row['record_id']}",
"report_date": row["_report_date"],
"user_id": row["user_id"],
"ad_type": "reward_video",
"feed_scene": row.get("feed_scene"),
"app_env": row.get("app_env"),
"our_code_id": row.get("our_code_id"),
"created_at": row["created_at"],
"hour": _cn_hour(row["created_at"]) if by_hour else None,
"has_impression": False,
"impressions": 0,
"ecpm": row["ecpm"],
"revenue_yuan": 0.0,
"adn": None,
"slot_id": None,
"has_reward": True,
"status": row["status"],
"expected_coin": int(row["expected_coin"]),
"actual_coin": int(row["actual_coin"]),
"matched": bool(row["matched"]),
"reward_detail": _reward_detail(row),
"sub_rewards": [],
"sub_count": 1,
})
else:
# 聚合单位 = 一次完整比价/领券流程:优先用 trace_id(比价带 comparisonTraceId、领券带 sessionTraceId,
# 整个流程不变;即使中途点广告致浮层关闭重弹、ad_session_id 变了,trace_id 仍不变 → 全流程聚成一行)。
# 无 trace_id(历史领券未上报 / 旧数据)回退整场 ad_session_id;再无则 record_id 各自成组、不误并。
grp_key = row.get("trace_id") or row.get("ad_session_id") or f"_rid-{row['record_id']}"
feed_groups.setdefault((row["user_id"], grp_key), []).append(row)
# 信息流分组 → 「一次比价 / 一次领券」父事件(收益恒 0:收益只算展示侧,避免与展示行重复计)。
for (uid, grp_key), group in feed_groups.items():
group.sort(key=lambda r: (r["created_at"], r["record_id"]))
rep = group[-1] # 代表条(最新一条):时间/场景/应用/代码位取它
expected_sum = sum(int(g["expected_coin"]) for g in group)
actual_sum = sum(int(g["actual_coin"]) for g in group)
# 父行 eCPM:组内各条 eCPM(分)均值(展示用,各条不同);无有效值则取代表条
ecpm_fens = [rewards.parse_ecpm_fen(g["ecpm"]) for g in group if g.get("ecpm")]
avg_ecpm = str(round(sum(ecpm_fens) / len(ecpm_fens))) if ecpm_fens else rep.get("ecpm")
# 主表逐行显示用:这次发奖广告的预估收益之和(发奖侧 eCPM 折算,钳顶同展示侧)。只放进
# row_revenue_yuan 给主表逐行展示,不进 revenue_yuan/合计/趋势——避免与展示侧 total 重复计。
row_revenue = round(sum(
min(rewards.parse_ecpm_yuan(g["ecpm"]), rewards.AD_ECPM_MAX_FEN / 100.0) / 1000.0
for g in group if g.get("ecpm")
), 6)
events.append({
"event_key": f"rwd-{row['record_id']}",
"report_date": row["_report_date"],
"user_id": row["user_id"],
"ad_type": _event_ad_type(row),
"feed_scene": row.get("feed_scene"),
"app_env": row.get("app_env"),
"our_code_id": row.get("our_code_id"),
"created_at": row["created_at"],
"hour": _cn_hour(row["created_at"]) if by_hour else None,
"event_key": f"feedgrp-{uid}-{grp_key}",
"report_date": rep["_report_date"],
"user_id": uid,
"ad_type": "draw", # 业务已全切 Draw 信息流,聚合行统一 draw
"feed_scene": rep.get("feed_scene"),
"app_env": rep.get("app_env"),
"our_code_id": rep.get("our_code_id"),
"created_at": rep["created_at"],
"hour": _cn_hour(rep["created_at"]) if by_hour else None,
"has_impression": False,
"impressions": 0,
"ecpm": row["ecpm"],
"ecpm": avg_ecpm,
"revenue_yuan": 0.0,
"row_revenue_yuan": row_revenue,
"adn": None,
"slot_id": None,
"has_reward": True,
"status": row["status"],
"expected_coin": int(row["expected_coin"]),
"actual_coin": int(row["actual_coin"]),
"matched": bool(row["matched"]),
"reward_detail": _reward_detail(row),
"status": rep["status"], # 代表状态(逐条见展开)
"expected_coin": expected_sum,
"actual_coin": actual_sum,
"matched": all(bool(g["matched"]) for g in group),
"reward_detail": None,
"sub_rewards": [_reward_detail(g) for g in group],
"sub_count": len(group),
})
# 「场景」作为全局筛选(与 user_id/ad_type 一致):同时作用于明细、合计与 daily/hourly 趋势。
@@ -212,6 +277,14 @@ def ad_revenue_report(
if feed_scene is not None:
events = [e for e in events if e.get("feed_scene") == feed_scene]
# app_env 过滤(2026-06-29 新增能力,修隐患:测试应用上报的假 eCPM 如 ¥678 CPM 会污染正式收益合计/平均):
# 显式传 "prod"/"test" 只看该环境;不传=全部(维持现状)。**不擅自把默认改成排除 test**——本地 dev 库多为
# test 数据、默认排除会使本地报表空,且「正式报表是否含 test」属产品口径。建议前端报表页加 app_env 筛选器
# (默认选 prod),或产品确认后再把默认改成排除 test。注:穿山甲后台收益列(total_pangle_*)暂未联动此过滤
# (它是独立对照列,且 pangle 的 test 是真实小额、非客户端那种假值)。
if app_env is not None:
events = [e for e in events if e.get("app_env") == app_env]
# 排序:time=按时间倒序(新→旧);ecpm=按 eCPM 数值倒序(eCPM 原值是字符串「分」,转数值排;
# 纯发奖行用其发奖采用的 eCPM,缺失/非法计 0 排末尾)。
if sort == "ecpm":
@@ -255,6 +328,25 @@ def ad_revenue_report(
for d in sorted(daily_map.values(), key=lambda x: x["date"])
]
# 穿山甲后台收益(GroMore 数据 API,T+1 入库 ad_pangle_daily_revenue):汇总 + 按天趋势级展示,
# 与上面客户端自报 eCPM 折算的预估并列对照(看 gap)。穿山甲数据**无用户/场景/类型维度**,故仅在
# 「全量视图」(未按 user_id / ad_type / feed_scene 过滤)给值;一旦带这些过滤,穿山甲数无法对应口径
# → 置 None,前端显示「-」并提示。逐条事件行不动(仍是客户端预估)。
pangle_filterable = user_id is None and ad_type is None and feed_scene is None
total_pangle_revenue_yuan: float | None = None
total_pangle_api_revenue_yuan: float | None = None
if pangle_filterable:
pangle_aggs = ad_pangle_revenue.aggregate_by_date(db, date_from=date_from, date_to=date_to)
if pangle_aggs:
by_date = {a["date"]: a for a in pangle_aggs}
for d in daily:
pa = by_date.get(d["date"])
d["pangle_revenue_yuan"] = pa["revenue_yuan"] if pa else None
d["pangle_api_revenue_yuan"] = pa["api_revenue_yuan"] if pa else None
total_pangle_revenue_yuan = round(sum(a["revenue_yuan"] for a in pangle_aggs), 6)
api_vals = [a["api_revenue_yuan"] for a in pangle_aggs if a["api_revenue_yuan"] is not None]
total_pangle_api_revenue_yuan = round(sum(api_vals), 6) if api_vals else None
# 按小时汇总(全量,不受分页 limit/offset 影响):供前端按小时趋势图(单日 granularity=hour 时用)。
# 只在 by_hour 下聚合(此时每个 event 带 hour);否则空。前端按天趋势仍用 daily。
hourly: list[dict] = []
@@ -298,11 +390,26 @@ def ad_revenue_report(
is_today = date_from == date_to == rewards.cn_today().isoformat()
dau = admin_stats.today_dau(db) if is_today else None
# 主表「逐行」= 单次广告行为(2026-07 按「一次比价/领券放一块」聚合):激励视频 = 一次观看一行(展示+发奖
# 按 ad_session_id 合并);一次比价 / 一次领券 = 该次整场多条广告按 ad_session_id 聚成一行(展开看逐条)。
# 信息流(draw/feed)的逐条展示(ad_ecpm,impressionId 各自独立、与整场发奖无公共键)不再单独占行
# ——其展示数 / eCPM / 预估收益已计入上面的全量统计(total_*、daily / hourly、type_stats、穿山甲对照),
# 只是主表不逐条铺开;逐条明细在父行展开里看(sub_rewards)。合计 / 趋势 / 分类大盘均基于全量 events,
# 不受此过滤影响;total / 分页只作用于主表行。
main_rows = [
e for e in events
if not (e["ad_type"] in ("draw", "feed") and e["has_impression"] and not e["has_reward"])
]
return {
"total": len(events),
"truncated": len(events) > offset + limit,
"total": len(main_rows),
"truncated": len(main_rows) > offset + limit,
"total_impressions": total_impressions,
"total_revenue_yuan": total_revenue_yuan,
# 穿山甲后台收益合计(元):预估 revenue + 收益Api;非全量视图(带 user/类型/场景过滤)或无数据为 None。
"total_pangle_revenue_yuan": total_pangle_revenue_yuan,
"total_pangle_api_revenue_yuan": total_pangle_api_revenue_yuan,
"pangle_revenue_available": total_pangle_revenue_yuan is not None,
"total_expected_coin": total_expected_coin,
"total_actual_coin": total_actual_coin,
"mismatch_count": mismatch_count,
@@ -310,5 +417,5 @@ def ad_revenue_report(
"hourly": hourly,
"type_stats": type_stats,
"dau": dau,
"items": events[offset:offset + limit],
"items": main_rows[offset:offset + limit],
}
+225
View File
@@ -0,0 +1,225 @@
"""admin「领券数据」看板聚合:发起/完成数、耗时均值与分位、按天/小时趋势、逐条明细。
数据源 coupon_session(一次领券一行,客户端 /api/v1/coupon/session 两段上报)。量级不大,全量拉
区间数据后 Python 聚合(分位 SQLite 无 percentile,统一 Python 算,PG 上也一致)。
- 发起数 = 区间内全部 session(含 started/completed/failed/abandoned),= 流失统计的基数。
- 完成数 / 耗时均值 / 分位 = 仅 status==completed 子集(成功跑完才有可比的"领券耗时")。
- summary/daily/hourly/total 在全量上算,不受分页;items 为排序后当前页。
"""
from __future__ import annotations
from datetime import UTC, date as _date, datetime
from sqlalchemy import func, or_, select
from sqlalchemy.orm import Session
from app.core import rewards
from app.models.coupon_state import CouponSession
from app.models.user import User
def _cn_hour(dt: datetime) -> int:
"""started_at(UTC 口径)→ 北京时间小时(023)。naive 当 UTC(sqlite),tz-aware 直接换算(pg)。"""
if dt.tzinfo is None:
dt = dt.replace(tzinfo=UTC)
return dt.astimezone(rewards.CN_TZ).hour
def _percentile(sorted_vals: list[int], q: float) -> int | None:
"""线性插值分位(q=0..100,numpy 默认法)。sorted_vals 须已升序;空返回 None。"""
if not sorted_vals:
return None
if len(sorted_vals) == 1:
return sorted_vals[0]
idx = (len(sorted_vals) - 1) * q / 100.0
lo = int(idx)
hi = min(lo + 1, len(sorted_vals) - 1)
frac = idx - lo
return round(sorted_vals[lo] * (1 - frac) + sorted_vals[hi] * frac)
def _avg(vals: list[int]) -> int | None:
return round(sum(vals) / len(vals)) if vals else None
def _session_to_row(r, phone: str | None = None, nickname: str | None = None) -> dict:
"""CouponSession ORM → 明细行 dict(主表「领券数据」与「用户全部领券」抽屉共用)。"""
return {
"id": r.id,
"trace_id": r.trace_id,
"user_id": r.user_id,
"user_phone": phone,
"user_nickname": nickname,
"status": r.status,
"platforms": r.platforms,
"origin_package": r.origin_package,
"elapsed_ms": r.elapsed_ms,
"platform_elapsed": r.platform_elapsed,
"device_model": r.device_model,
"rom": r.rom,
"app_env": r.app_env,
"started_at": r.started_at,
"claimed_count": r.claimed_count,
"trace_url": r.trace_url,
}
def _empty_result() -> dict:
return {
"summary": {
"started_count": 0, "completed_count": 0, "avg_elapsed_ms": None,
"p5_ms": None, "p50_ms": None, "p95_ms": None, "p99_ms": None,
},
"daily": [],
"hourly": [],
"total": 0,
"items": [],
}
def coupon_data_report(
db: Session,
*,
date_from: str,
date_to: str,
user: str | None = None,
app_env: str | None = None,
granularity: str = "day",
limit: int = 500,
offset: int = 0,
sort: str = "time",
) -> dict:
"""日期区间(北京自然日 started_date,闭区间)领券数据:汇总卡 + 趋势 + 逐条明细。
- user:手机号/昵称模糊搜(匹配不到任何用户 → 空结果)。
- app_env:prod/dev 精确;None=全部。
- sort:time=发起时刻倒序(默认) / elapsed=全程耗时倒序(None 末尾)。
"""
by_hour = granularity == "hour"
d_from = _date.fromisoformat(date_from)
d_to = _date.fromisoformat(date_to)
# user 模糊 → 先定位匹配用户 id;匹配不到直接空结果(不全表扫)。
user_ids: set[int] | None = None
if user:
like = f"%{user}%"
user_ids = set(db.execute(
select(User.id).where(or_(User.phone.like(like), User.nickname.like(like)))
).scalars().all())
if not user_ids:
return _empty_result()
stmt = select(CouponSession).where(
CouponSession.started_date >= d_from,
CouponSession.started_date <= d_to,
)
if app_env is not None:
stmt = stmt.where(CouponSession.app_env == app_env)
if user_ids is not None:
stmt = stmt.where(CouponSession.user_id.in_(user_ids))
rows = list(db.execute(stmt).scalars())
# ── 汇总卡 ──
completed_elapsed = sorted(
r.elapsed_ms for r in rows if r.status == "completed" and r.elapsed_ms is not None
)
summary = {
"started_count": len(rows),
"completed_count": sum(1 for r in rows if r.status == "completed"),
"avg_elapsed_ms": _avg(completed_elapsed),
"p5_ms": _percentile(completed_elapsed, 5),
"p50_ms": _percentile(completed_elapsed, 50),
"p95_ms": _percentile(completed_elapsed, 95),
"p99_ms": _percentile(completed_elapsed, 99),
}
# ── 按天趋势(柱=发起/完成数,线=平均耗时)──
daily_map: dict[str, dict] = {}
for r in rows:
d = r.started_date.isoformat()
b = daily_map.get(d)
if b is None:
b = {"date": d, "started_count": 0, "completed_count": 0, "_elapsed": []}
daily_map[d] = b
b["started_count"] += 1
if r.status == "completed":
b["completed_count"] += 1
if r.elapsed_ms is not None:
b["_elapsed"].append(r.elapsed_ms)
daily = [
{
"date": b["date"],
"started_count": b["started_count"],
"completed_count": b["completed_count"],
"avg_elapsed_ms": _avg(b["_elapsed"]),
}
for b in sorted(daily_map.values(), key=lambda x: x["date"])
]
# ── 按小时趋势(单日 hour 粒度)──
hourly: list[dict] = []
if by_hour:
hour_map: dict[int, dict] = {}
for r in rows:
h = _cn_hour(r.started_at)
b = hour_map.get(h)
if b is None:
b = {"hour": h, "started_count": 0, "completed_count": 0, "_elapsed": []}
hour_map[h] = b
b["started_count"] += 1
if r.status == "completed":
b["completed_count"] += 1
if r.elapsed_ms is not None:
b["_elapsed"].append(r.elapsed_ms)
hourly = [
{
"hour": b["hour"],
"started_count": b["started_count"],
"completed_count": b["completed_count"],
"avg_elapsed_ms": _avg(b["_elapsed"]),
}
for b in sorted(hour_map.values(), key=lambda x: x["hour"])
]
# ── 明细:排序 + 分页 + 补用户手机号/昵称(批量,防 N+1)──
if sort == "elapsed":
rows.sort(key=lambda r: (r.elapsed_ms is None, -(r.elapsed_ms or 0)))
else: # time:发起时刻倒序
rows.sort(key=lambda r: r.started_at, reverse=True)
page = rows[offset:offset + limit]
uids = {r.user_id for r in page if r.user_id is not None}
user_map: dict[int, tuple[str | None, str | None]] = {}
if uids:
user_map = {
uid: (phone, nickname)
for uid, phone, nickname in db.execute(
select(User.id, User.phone, User.nickname).where(User.id.in_(uids))
).all()
}
items = []
for r in page:
phone, nickname = user_map.get(r.user_id, (None, None)) if r.user_id is not None else (None, None)
items.append(_session_to_row(r, phone, nickname))
return {
"summary": summary,
"daily": daily,
"hourly": hourly,
"total": len(rows),
"items": items,
}
def coupon_user_records(db: Session, *, user_id: int, limit: int = 100) -> dict:
"""某用户全部领券记录(点手机号抽屉用):按发起时刻倒序、不限日期,total=该用户领券总次数。"""
rows = list(db.execute(
select(CouponSession)
.where(CouponSession.user_id == user_id)
.order_by(CouponSession.started_at.desc())
.limit(limit)
).scalars())
total = db.execute(
select(func.count()).select_from(CouponSession).where(CouponSession.user_id == user_id)
).scalar_one()
return {"items": [_session_to_row(r) for r in rows], "total": int(total)}
+166 -1
View File
@@ -7,13 +7,14 @@ from __future__ import annotations
from datetime import datetime, timedelta, timezone
from decimal import Decimal, InvalidOperation
from typing import Any
from uuid import uuid4
from sqlalchemy import desc, func, select
from sqlalchemy.orm import Session
from app.admin.repositories.queries import _as_utc, offset_paginate
from app.integrations import meituan
from app.integrations import jd_union, meituan
from app.repositories import cps_link as cps_link_repo
from app.models.cps_activity import CpsActivity
from app.models.cps_group import CpsGroup
@@ -24,6 +25,11 @@ from app.models.cps_wx_user import CpsWxUser
# 美团订单状态:取消(4)/风控(5)不计佣金;结算(6)为佣金真正到账
_INVALID_STATUS = {"4", "5"}
_SETTLED_STATUS = "6"
_JD_INVALID_CODES = {
"2", "3", "4", "5", "6", "7", "8", "9", "11", "13", "14", "19", "20", "21",
"22", "23", "25", "26", "27", "28", "29", "30", "31", "34", "35", "36",
}
_JD_UNPAID_CODES = {"15"}
# CPS 点击时序按北京时区分桶(运营看的是北京时间)
_BJ_TZ = timezone(timedelta(hours=8))
@@ -47,6 +53,36 @@ def _ts_to_dt(ts: object) -> datetime | None:
"""秒级时间戳 → tz-aware UTC datetime(绝对时刻,前端按北京展示)。"""
if not ts:
return None
def _jd_dt_to_utc(value: object) -> datetime | None:
"""京东时间字符串(北京时间) → UTC aware datetime。"""
if value is None:
return None
s = str(value).strip()
if not s:
return None
for fmt in ("%Y-%m-%d %H:%M:%S", "%Y-%m-%d"):
try:
dt = datetime.strptime(s, fmt)
return dt.replace(tzinfo=_BJ_TZ).astimezone(timezone.utc)
except ValueError:
continue
return None
def _text(value: object) -> str | None:
if value is None:
return None
s = str(value).strip()
return s or None
def _pick(row: dict[str, Any], *keys: str) -> Any:
for key in keys:
if key in row and row[key] is not None:
return row[key]
return None
try:
return datetime.fromtimestamp(int(ts), tz=timezone.utc)
except (ValueError, OSError, TypeError):
@@ -249,6 +285,80 @@ def _map_order_fields(r: dict) -> dict:
}
def _jd_order_key(r: dict[str, Any]) -> str | None:
row_id = _text(_pick(r, "id", "rowId", "orderRowId"))
if row_id:
return f"jd:{row_id}"
order_id = _text(_pick(r, "orderId", "parentOrderId"))
sku_id = _text(_pick(r, "skuId"))
if order_id and sku_id:
return f"jd:{order_id}:{sku_id}"
if order_id:
return f"jd:{order_id}"
return None
def _map_jd_order_fields(r: dict[str, Any]) -> dict:
"""京东 order.row.query 单条订单行 → CpsOrder 字段。"""
sku_name = _text(_pick(r, "skuName", "goodsName", "productName"))
if sku_name and len(sku_name) > 500:
sku_name = sku_name[:500]
valid_code = _text(_pick(r, "validCode", "valid_code"))
actual_fee = _yuan_to_cents(_pick(r, "actualFee", "actual_fee"))
estimate_fee = _yuan_to_cents(_pick(r, "estimateFee", "estimate_fee"))
commission = actual_fee if actual_fee not in (None, 0) else estimate_fee
order_time = _jd_dt_to_utc(_pick(r, "orderTime", "order_time"))
return {
"platform": "jd",
"external_order_id": _text(_pick(r, "orderId", "parentOrderId")),
"external_row_id": _text(_pick(r, "id", "rowId", "orderRowId")),
"sid": _text(_pick(r, "subUnionId", "sub_union_id")),
"act_id": None,
"biz_line": None,
"trade_type": None,
"pay_price_cents": _yuan_to_cents(
_pick(r, "actualCosPrice", "estimateCosPrice", "price")
),
"commission_cents": commission,
"commission_rate": _text(_pick(r, "commissionRate", "commission_rate")),
"refund_price_cents": None,
"refund_profit_cents": None,
"estimated_commission_cents": estimate_fee,
"actual_commission_cents": actual_fee,
"mt_status": None,
"jd_valid_code": valid_code,
"invalid_reason": None if _is_jd_valid_code(valid_code) else f"validCode={valid_code}",
"product_name": sku_name,
"settle_month": _text(_pick(r, "payMonth", "settleMonth", "pay_month")),
"site_id": _text(_pick(r, "siteId", "site_id")),
"position_id": _text(_pick(r, "positionId", "position_id")),
"pid": _text(_pick(r, "pid")),
"sub_union_id": _text(_pick(r, "subUnionId", "sub_union_id")),
"pay_time": order_time,
"mt_update_time": _jd_dt_to_utc(_pick(r, "modifyTime", "updateTime", "modify_time"))
or order_time,
"raw": r,
}
def _is_jd_valid_code(valid_code: str | None) -> bool:
code = str(valid_code).strip() if valid_code is not None else ""
return bool(code and code not in _JD_INVALID_CODES and code not in _JD_UNPAID_CODES)
def is_jd_order_valid(order: CpsOrder) -> bool:
return _is_jd_valid_code(order.jd_valid_code)
def effective_commission_cents(order: CpsOrder) -> int:
if order.platform == "jd":
if order.actual_commission_cents not in (None, 0):
return order.actual_commission_cents or 0
if order.estimated_commission_cents is not None:
return order.estimated_commission_cents or 0
return order.commission_cents or 0
def reconcile_orders(
db: Session, *, start_time: int, end_time: int,
query_time_type: int = 1, sid: str | None = None, max_pages: int = 200,
@@ -274,6 +384,11 @@ def reconcile_orders(
continue
fetched += 1
fields = _map_order_fields(r)
fields.setdefault("platform", "meituan")
fields.setdefault("external_order_id", order_id)
fields.setdefault("external_row_id", None)
fields.setdefault("estimated_commission_cents", fields.get("commission_cents"))
fields.setdefault("actual_commission_cents", None)
existing = db.execute(
select(CpsOrder).where(CpsOrder.order_id == order_id)
).scalar_one_or_none()
@@ -291,6 +406,56 @@ def reconcile_orders(
return {"fetched": fetched, "inserted": inserted, "updated": updated, "pages": pages}
def reconcile_jd_orders(
db: Session, *, start_time: datetime, end_time: datetime,
query_time_type: int = 3, max_pages: int = 100,
) -> dict:
"""调京东 order.row.query 拉单 → 按订单行 upsert。
京东单次查询窗口最多 1 小时,这里按北京自然时间切窗并逐页拉取。
"""
fetched = inserted = updated = pages = 0
cur = start_time
while cur < end_time:
win_end = min(cur + timedelta(hours=1), end_time)
page = 1
while page <= max_pages:
resp = jd_union.query_order_rows(
start_time=cur,
end_time=win_end,
query_time_type=query_time_type,
page_index=page,
page_size=200,
)
rows = resp.get("rows") or []
has_more = bool(resp.get("has_more"))
if not rows:
break
pages += 1
for r in rows:
order_id = _jd_order_key(r)
if not order_id:
continue
fetched += 1
fields = _map_jd_order_fields(r)
existing = db.execute(
select(CpsOrder).where(CpsOrder.order_id == order_id)
).scalar_one_or_none()
if existing is None:
db.add(CpsOrder(order_id=order_id, **fields))
inserted += 1
else:
for k, v in fields.items():
setattr(existing, k, v)
updated += 1
if not has_more or len(rows) < 200:
break
page += 1
cur = win_end
db.commit()
return {"fetched": fetched, "inserted": inserted, "updated": updated, "pages": pages}
def list_orders(
db: Session, *, sid: str | None = None, mt_status: str | None = None,
limit: int = 20, cursor: int | None = None,
+90 -8
View File
@@ -125,8 +125,9 @@ def list_users(
return offset_paginate(db, stmt, (order_fn(sort_col), id_order), limit=limit, cursor=cursor)
def _attach_user_info(db: Session, records: list[ComparisonRecord]) -> None:
"""给每条比价记录瞬态挂 phone/nickname(非 DB 列,供 admin schema from_attributes 读)。"""
def _attach_user_info(db: Session, records: list[ComparisonRecord | Feedback | PriceReport]) -> None:
"""给每条记录瞬态挂 phone/nickname(非 DB 列,供 admin schema from_attributes 读)。
按 user_id 鸭子类型,比价记录/反馈/上报通用。"""
uids = {r.user_id for r in records}
if not uids:
return
@@ -562,7 +563,25 @@ def list_feedbacks(
sort_col = sort_cols.get(sort_by, Feedback.id)
order_fn = asc if sort_order == "asc" else desc
id_order = asc(Feedback.id) if sort_order == "asc" else desc(Feedback.id)
return offset_paginate(db, stmt, (order_fn(sort_col), id_order), limit=limit, cursor=cursor)
items, next_cursor, total = offset_paginate(
db, stmt, (order_fn(sort_col), id_order), limit=limit, cursor=cursor
)
_attach_user_info(db, items) # 列表展示完整手机号(点手机号查该用户全部反馈)
return items, next_cursor, total
def feedback_summary(db: Session) -> dict:
"""反馈审核台各状态计数(待审核/已采纳/未采纳/合计)。待审核含历史 new 态(与前端 isPending 一致)。"""
rows = db.execute(
select(Feedback.status, func.count(Feedback.id)).group_by(Feedback.status)
).all()
by_status = {status: int(count) for status, count in rows}
return {
"pending": by_status.get("pending", 0) + by_status.get("new", 0),
"adopted": by_status.get("adopted", 0),
"rejected": by_status.get("rejected", 0),
"total": sum(by_status.values()),
}
def list_analytics_events(
@@ -769,6 +788,7 @@ def get_user_overview(db: Session, user_id: int) -> dict | None:
"user": user,
"coin_balance": acc.coin_balance if acc else 0,
"cash_balance_cents": acc.cash_balance_cents if acc else 0,
"invite_cash_balance_cents": acc.invite_cash_balance_cents if acc else 0,
"total_coin_earned": acc.total_coin_earned if acc else 0,
"comparison_total": _count(ComparisonRecord, ComparisonRecord.user_id == user_id),
"comparison_success": _count(
@@ -786,6 +806,12 @@ def get_user_overview(db: Session, user_id: int) -> dict | None:
}
def _as_utc_naive(value: datetime) -> datetime:
"""窗口入参 → UTC naive(= _as_utc 去时区),与库里按 naive UTC 存取的 created_at 同口径比较。
历史遗留:_window_conds 一直引用本函数却未定义(自定义区间会 NameError),此处补上。"""
return _as_utc(value).replace(tzinfo=None)
def _window_conds(col, date_from: datetime | None, date_to: datetime | None) -> list:
"""把 [date_from, date_to] 转成对 col(created_at)的过滤条件;都为 None = 全量(注册至今)。"""
conds = []
@@ -875,6 +901,13 @@ def user_reward_stats(
}
def _cn_wall_to_utc(dt: datetime) -> datetime:
"""coin_transaction 存的是北京 wall-clock(naive,见 wallet.grant_coins「存北京 wall-clock」),转成 UTC naive,
与广告表(func.now() UTC)统一 —— 让本函数按同一绝对时刻排序、且前端 apiTime(把无时区时间当 UTC 再 +8 展示)
口径一致;否则签到会比实际多显示 8 小时(北京时间又被 +8)。"""
return dt.replace(tzinfo=rewards.CN_TZ).astimezone(timezone.utc).replace(tzinfo=None)
def user_coin_records(
db: Session,
user_id: int,
@@ -895,6 +928,9 @@ def user_coin_records(
offset = max(cursor or 0, 0)
fetch = offset + limit + 1
rows: list[dict] = []
# coin_transaction 存北京 wall-clock(其余表存 UTC);签到窗口边界 +8h 对齐北京,过滤/计数才不偏移 8 小时
signin_from = date_from + timedelta(hours=8) if date_from is not None else None
signin_to = date_to + timedelta(hours=8) if date_to is not None else None
for rec in db.execute(
select(AdRewardRecord)
@@ -938,7 +974,7 @@ def user_coin_records(
.where(
CoinTransaction.user_id == user_id,
CoinTransaction.biz_type == "signin",
*_window_conds(CoinTransaction.created_at, date_from, date_to),
*_window_conds(CoinTransaction.created_at, signin_from, signin_to),
)
.order_by(CoinTransaction.created_at.desc())
.limit(fetch)
@@ -946,7 +982,8 @@ def user_coin_records(
rows.append({
"source": "signin",
"source_label": "签到",
"created_at": rec.created_at,
# 北京 wall-clock → UTC,与广告记录统一(前端 apiTime 会 +8 回北京展示,不然签到会多 8 小时)
"created_at": _cn_wall_to_utc(rec.created_at),
"ecpm": None,
"coin": rec.amount,
})
@@ -972,27 +1009,72 @@ def user_coin_records(
+ _count(
CoinTransaction, CoinTransaction.user_id == user_id,
CoinTransaction.biz_type == "signin",
*_window_conds(CoinTransaction.created_at, date_from, date_to),
*_window_conds(CoinTransaction.created_at, signin_from, signin_to),
)
)
return rows[offset:offset + limit], (offset + limit if has_more else None), total
def _attach_price_report_comparison(db: Session, records: list[PriceReport]) -> None:
"""给每条上报瞬态挂关联比价记录的 trace + 设备/版本快照:
trace_id/trace_url(点 Trace 看完整比价过程)、device_model/rom_name/android_version(机型OS版本列)、
app_version(提交版本号列,= 提交时我们 app 的 versionName)。
comparison_record_id 为空 / 关联记录查不到 → 全 None。"""
cids = {r.comparison_record_id for r in records if r.comparison_record_id is not None}
rows = (
db.execute(
select(
ComparisonRecord.id,
ComparisonRecord.trace_id,
ComparisonRecord.trace_url,
ComparisonRecord.device_model,
ComparisonRecord.rom_name,
ComparisonRecord.android_version,
ComparisonRecord.app_version,
).where(ComparisonRecord.id.in_(cids))
).all()
if cids
else []
)
cmap = {row.id: row for row in rows}
for r in records:
row = cmap.get(r.comparison_record_id)
r.trace_id = row.trace_id if row else None
r.trace_url = row.trace_url if row else None
r.device_model = row.device_model if row else None
r.rom_name = row.rom_name if row else None
r.android_version = row.android_version if row else None
r.app_version = row.app_version if row else None
def list_price_reports(
db: Session,
*,
status: str | None = None,
user_id: int | None = None,
sort_by: str = "id",
sort_order: str = "desc",
limit: int = 20,
cursor: int | None = None,
) -> tuple[list[PriceReport], int | None, int]:
"""上报更低价列表(admin 全量,可按状态/用户筛)。offset 分页 + total,id 倒序。"""
"""上报更低价列表(admin 全量,可按状态/用户筛)。offset 分页 + total,按 id·提交时间排序。
join User 取 phone 瞬态挂(列表展示完整手机号);按 comparison_record_id 挂 trace_id/trace_url。"""
stmt = select(PriceReport)
if status:
stmt = stmt.where(PriceReport.status == status)
if user_id is not None:
stmt = stmt.where(PriceReport.user_id == user_id)
return offset_paginate(db, stmt, (PriceReport.id.desc(),), limit=limit, cursor=cursor)
sort_cols = {"id": PriceReport.id, "created_at": PriceReport.created_at}
sort_col = sort_cols.get(sort_by, PriceReport.id)
order_fn = asc if sort_order == "asc" else desc
id_order = asc(PriceReport.id) if sort_order == "asc" else desc(PriceReport.id)
items, next_cursor, total = offset_paginate(
db, stmt, (order_fn(sort_col), id_order), limit=limit, cursor=cursor
)
_attach_user_info(db, items)
_attach_price_report_comparison(db, items)
return items, next_cursor, total
def price_report_summary(db: Session) -> dict:
+418 -14
View File
@@ -5,20 +5,44 @@ user.last_login_at / comparison_record.status / withdraw_order.status)要加索
"""
from __future__ import annotations
from datetime import datetime, timedelta, timezone
from datetime import date, datetime, time, timedelta, timezone
from decimal import Decimal, InvalidOperation
from sqlalchemy import func, select
from sqlalchemy.orm import Session
from app.models.ad_feed_reward import AdFeedRewardRecord
from app.models.ad_reward import AdRewardRecord
from app.models.analytics_event import AnalyticsEvent
from app.models.comparison import ComparisonRecord
from app.models.coupon_state import CouponPromptEngagement
from app.models.cps_order import CpsOrder
from app.models.feedback import Feedback
from app.models.savings import SavingsRecord
from app.models.signin import SigninBoostRecord, SigninRecord
from app.models.user import User
from app.models.wallet import CoinTransaction, WithdrawOrder
_BEIJING = timezone(timedelta(hours=8))
COUPON_REWARD_BIZ_TYPES = ("reward_video", "ad_reward", "coupon", "coupon_reward")
COMPARISON_REWARD_BIZ_TYPES = ("comparison", "compare_reward", "comparison_reward")
EXCLUDED_REWARD_BIZ_TYPES = ("invite_inviter", "invite_invitee", "admin_grant")
UNCLASSIFIED_FEED_BIZ_TYPES = ("feed_ad_reward",)
REGULAR_TASK_EXCLUDED_BIZ_TYPES = (
*COUPON_REWARD_BIZ_TYPES,
*COMPARISON_REWARD_BIZ_TYPES,
*EXCLUDED_REWARD_BIZ_TYPES,
*UNCLASSIFIED_FEED_BIZ_TYPES,
)
MEITUAN_CPS_INVALID_STATUSES = ("4", "5")
MEITUAN_CPS_SETTLED_STATUS = "6"
COMPARE_START_EVENT = "real_compare_start"
COUPON_START_EVENT = "real_coupon_start"
JD_CPS_INVALID_CODES = {
"2", "3", "4", "5", "6", "7", "8", "9", "11", "13", "14", "19", "20", "21",
"22", "23", "25", "26", "27", "28", "29", "30", "31", "34", "35", "36",
}
JD_CPS_UNPAID_CODES = {"15"}
def _beijing_today_start_utc() -> datetime:
@@ -29,32 +53,129 @@ def _beijing_today_start_utc() -> datetime:
def today_dau(db: Session) -> int:
"""今日活跃用户数(DAU):北京时区今天 0 点后登录过(last_login_at)
"""今日活跃用户数(DAU):登录 + 开始比价 + 开始领券,按用户去重
大盘与广告收益报表共用此口径,单一来源避免漂移
⚠️ last_login_at 是单值字段(只存最后一次登录时刻),故只能算「今日」,
无法回溯历史某天的 DAU——调用方按此约束决定历史区间是否展示。
广告收益报表复用这个函数;历史窗口 DAU 由 dashboard_overview 的 period 口径另算
"""
today_bj = datetime.now(_BEIJING).date()
today_start = _beijing_today_start_utc()
return db.execute(
select(func.count(User.id)).where(User.last_login_at >= today_start)
).scalar_one()
tomorrow_start = today_start + timedelta(days=1)
login_user_ids = _id_set(
db,
select(User.id).where(User.last_login_at >= today_start, User.last_login_at < tomorrow_start),
)
compare_start_user_ids = _event_user_ids(
db, (COMPARE_START_EVENT,), today_start, tomorrow_start
)
coupon_event_user_ids = _event_user_ids(
db, (COUPON_START_EVENT,), today_start, tomorrow_start
)
coupon_claim_user_ids = _id_set(
db,
select(CouponPromptEngagement.user_id).where(
CouponPromptEngagement.engage_date == today_bj,
CouponPromptEngagement.engage_type == "claim_started",
),
)
return len(login_user_ids | compare_start_user_ids | coupon_event_user_ids | coupon_claim_user_ids)
def dashboard_overview(db: Session) -> dict:
def _default_period_end() -> date:
"""新版大盘不含今日,默认窗口结束日=北京时间昨天。"""
return datetime.now(_BEIJING).date() - timedelta(days=1)
def _normalize_period(date_from: date | None, date_to: date | None) -> tuple[date, date]:
end = date_to or _default_period_end()
start = date_from or end
if start > end:
start, end = end, start
return start, end
def _period_bounds(date_from: date, date_to: date) -> tuple[datetime, datetime, datetime, datetime]:
"""返回同一北京自然日窗口的 UTC aware 边界和北京 naive 边界。
user.created_at / last_login_at 是 UTC aware 口径;比较/金币等历史上有北京 naive
写入,所以两套边界同时保留。
"""
start_bj = datetime.combine(date_from, time.min, tzinfo=_BEIJING)
end_bj = datetime.combine(date_to + timedelta(days=1), time.min, tzinfo=_BEIJING)
start_utc = start_bj.astimezone(timezone.utc)
end_utc = end_bj.astimezone(timezone.utc)
return (
start_utc,
end_utc,
start_bj.replace(tzinfo=None),
end_bj.replace(tzinfo=None),
)
def _date_range(date_from: date, date_to: date) -> list[date]:
days = (date_to - date_from).days
return [date_from + timedelta(days=i) for i in range(days + 1)]
def _id_set(db: Session, stmt) -> set[int]:
return {int(v) for v in db.execute(stmt).scalars().all() if v is not None}
def _event_user_ids(
db: Session, event_names: tuple[str, ...], start_utc: datetime, end_utc: datetime
) -> set[int]:
return _id_set(
db,
select(AnalyticsEvent.user_id).where(
AnalyticsEvent.user_id.is_not(None),
AnalyticsEvent.event.in_(event_names),
AnalyticsEvent.created_at >= start_utc,
AnalyticsEvent.created_at < end_utc,
),
)
def _commission_rate_percent(raw: str | None) -> Decimal | None:
"""美团 commissionRate 原值: "300"=3%, "10"=0.1%;也兼容 "3%""""
if raw is None:
return None
s = str(raw).strip()
if not s:
return None
try:
if s.endswith("%"):
return Decimal(s[:-1])
val = Decimal(s)
except (InvalidOperation, ValueError):
return None
return val / Decimal("100")
def _jd_valid_order(order: CpsOrder) -> bool:
code = str(order.jd_valid_code).strip() if order.jd_valid_code is not None else ""
return bool(code and code not in JD_CPS_INVALID_CODES and code not in JD_CPS_UNPAID_CODES)
def dashboard_overview(
db: Session, *, date_from: date | None = None, date_to: date | None = None
) -> dict:
today_start = _beijing_today_start_utc()
period_from, period_to = _normalize_period(date_from, date_to)
start_utc, end_utc, start_local, end_local = _period_bounds(period_from, period_to)
def _count(model, *conds) -> int:
stmt = select(func.count(model.id))
if conds:
stmt = stmt.where(*conds)
return db.execute(stmt).scalar_one()
return int(db.execute(stmt).scalar_one())
def _sum(col, *conds) -> int:
stmt = select(func.coalesce(func.sum(col), 0))
if conds:
stmt = stmt.where(*conds)
return db.execute(stmt).scalar_one()
return int(db.execute(stmt).scalar_one())
def _user_id_set(stmt) -> set[int]:
return {int(v) for v in db.execute(stmt).scalars().all() if v is not None}
# ===== 用户 =====
by_status = dict(
@@ -74,6 +195,223 @@ def dashboard_overview(db: Session) -> dict:
comparison_total = _count(ComparisonRecord)
comparison_success = _count(ComparisonRecord, ComparisonRecord.status == "success")
success_rate = round(comparison_success / comparison_total, 4) if comparison_total else 0.0
period_comparison_conds = (
ComparisonRecord.created_at >= start_local,
ComparisonRecord.created_at < end_local,
)
period_comparison_total = _count(ComparisonRecord, *period_comparison_conds)
period_comparison_success = _count(
ComparisonRecord,
*period_comparison_conds,
ComparisonRecord.status == "success",
)
period_comparison_success_rate = (
round(period_comparison_success / period_comparison_total, 4)
if period_comparison_total
else 0.0
)
period_saved_positive_count = _count(
ComparisonRecord,
*period_comparison_conds,
ComparisonRecord.status == "success",
ComparisonRecord.saved_amount_cents > 0,
)
period_saved_positive_sum = _sum(
ComparisonRecord.saved_amount_cents,
*period_comparison_conds,
ComparisonRecord.status == "success",
ComparisonRecord.saved_amount_cents > 0,
)
period_avg_saved_cents = (
round(period_saved_positive_sum / period_saved_positive_count)
if period_saved_positive_count
else None
)
period_avg_duration_ms = db.execute(
select(func.avg(ComparisonRecord.total_ms)).where(
*period_comparison_conds,
ComparisonRecord.total_ms.is_not(None),
ComparisonRecord.total_ms > 0,
)
).scalar_one()
period_avg_duration_ms = (
round(float(period_avg_duration_ms))
if period_avg_duration_ms is not None
else None
)
ordered_exists = (
select(SavingsRecord.id)
.where(
SavingsRecord.user_id == ComparisonRecord.user_id,
SavingsRecord.source == "compare",
SavingsRecord.shop_name.is_not(None),
SavingsRecord.shop_name == ComparisonRecord.store_name,
)
.exists()
)
period_ordered_count = _count(
ComparisonRecord,
*period_comparison_conds,
ComparisonRecord.store_name.is_not(None),
ordered_exists,
)
# ===== 日期窗口用户 =====
period_new_user_ids = _user_id_set(
select(User.id).where(User.created_at >= start_utc, User.created_at < end_utc)
)
login_user_ids = _user_id_set(
select(User.id).where(User.last_login_at >= start_utc, User.last_login_at < end_utc)
)
compare_start_user_ids = _event_user_ids(
db, (COMPARE_START_EVENT,), start_utc, end_utc
)
coupon_event_user_ids = _event_user_ids(
db, (COUPON_START_EVENT,), start_utc, end_utc
)
coupon_claim_user_ids = _user_id_set(
select(CouponPromptEngagement.user_id).where(
CouponPromptEngagement.engage_date >= period_from,
CouponPromptEngagement.engage_date <= period_to,
CouponPromptEngagement.engage_type == "claim_started",
)
)
period_active_user_ids = (
login_user_ids | compare_start_user_ids | coupon_event_user_ids | coupon_claim_user_ids
)
period_retained_new_user_ids = period_new_user_ids & period_active_user_ids
period_retention_rate = (
round(len(period_retained_new_user_ids) / len(period_new_user_ids), 4)
if period_new_user_ids
else None
)
trend_points: list[dict] = []
for cur_date in _date_range(period_from, period_to):
day_start_utc, day_end_utc, day_start_local, day_end_local = _period_bounds(
cur_date, cur_date
)
daily_comparison_conds = (
ComparisonRecord.created_at >= day_start_local,
ComparisonRecord.created_at < day_end_local,
)
daily_login_user_ids = _user_id_set(
select(User.id).where(
User.last_login_at >= day_start_utc,
User.last_login_at < day_end_utc,
)
)
daily_compare_start_user_ids = _event_user_ids(
db, (COMPARE_START_EVENT,), day_start_utc, day_end_utc
)
daily_coupon_event_user_ids = _event_user_ids(
db, (COUPON_START_EVENT,), day_start_utc, day_end_utc
)
daily_coupon_claim_user_ids = _user_id_set(
select(CouponPromptEngagement.user_id).where(
CouponPromptEngagement.engage_date == cur_date,
CouponPromptEngagement.engage_type == "claim_started",
)
)
trend_points.append(
{
"date": cur_date,
"active_users": len(
daily_login_user_ids
| daily_compare_start_user_ids
| daily_coupon_event_user_ids
| daily_coupon_claim_user_ids
),
"new_users": _count(
User,
User.created_at >= day_start_utc,
User.created_at < day_end_utc,
),
"comparisons": _count(ComparisonRecord, *daily_comparison_conds),
}
)
period_coin_conds = (
CoinTransaction.created_at >= start_local,
CoinTransaction.created_at < end_local,
CoinTransaction.amount > 0,
)
period_reward_video_coin_total = _sum(
CoinTransaction.amount,
*period_coin_conds,
CoinTransaction.biz_type.in_(("reward_video", "ad_reward")),
)
period_feed_ad_coin_total = _sum(
CoinTransaction.amount,
*period_coin_conds,
CoinTransaction.biz_type == "feed_ad_reward",
)
period_signin_coin_total = _sum(
CoinTransaction.amount,
*period_coin_conds,
CoinTransaction.biz_type == "signin",
)
period_signin_boost_coin_total = _sum(
CoinTransaction.amount,
*period_coin_conds,
CoinTransaction.biz_type == "signin_boost",
)
period_task_coin_total = _sum(
CoinTransaction.amount,
*period_coin_conds,
CoinTransaction.biz_type.like("task_%"),
)
period_coupon_reward_coin_total = _sum(
CoinTransaction.amount,
*period_coin_conds,
CoinTransaction.biz_type.in_(COUPON_REWARD_BIZ_TYPES),
)
period_comparison_reward_coin_total = _sum(
CoinTransaction.amount,
*period_coin_conds,
CoinTransaction.biz_type.in_(COMPARISON_REWARD_BIZ_TYPES),
)
period_regular_task_coin_total = _sum(
CoinTransaction.amount,
*period_coin_conds,
CoinTransaction.biz_type.notin_(REGULAR_TASK_EXCLUDED_BIZ_TYPES),
)
period_cps_orders = list(
db.execute(
select(CpsOrder).where(
CpsOrder.pay_time >= start_utc,
CpsOrder.pay_time < end_utc,
)
).scalars()
)
period_meituan_orders = [
o for o in period_cps_orders if (o.platform or "meituan") == "meituan"
]
period_meituan_valid_orders = [
o for o in period_meituan_orders if o.mt_status not in MEITUAN_CPS_INVALID_STATUSES
]
period_jd_orders = [o for o in period_cps_orders if o.platform == "jd"]
period_jd_valid_orders = [o for o in period_jd_orders if _jd_valid_order(o)]
period_jd_invalid_orders = [
o for o in period_jd_orders if o.jd_valid_code and not _jd_valid_order(o)
]
period_meituan_hit_count = 0
period_meituan_miss_count = 0
period_meituan_unknown_rate_count = 0
for order in period_meituan_valid_orders:
rate = _commission_rate_percent(order.commission_rate)
if rate is None:
period_meituan_unknown_rate_count += 1
elif rate < Decimal("1"):
period_meituan_miss_count += 1
else:
period_meituan_hit_count += 1
period_meituan_hit_denominator = period_meituan_hit_count + period_meituan_miss_count
period_meituan_hit_rate = (
round(period_meituan_hit_count / period_meituan_hit_denominator, 4)
if period_meituan_hit_denominator
else None
)
return {
"users": {
@@ -132,7 +470,73 @@ def dashboard_overview(db: Session) -> dict:
"success": comparison_success,
"success_rate": success_rate,
},
"feedback": {"new": _count(Feedback, Feedback.status.in_(("pending", "new")))},
# CPS 收入数据源未接(referral-link 只换链接,转化/佣金未回收)→ 前端显示"待接入"。
"cps": {"available": False, "note": "CPS 转化数据未接入(P2)"},
"period": {
"date_from": period_from,
"date_to": period_to,
"users": {
"new": len(period_new_user_ids),
"active": len(period_active_user_ids),
"retained_new_users": len(period_retained_new_user_ids),
"retention_rate": period_retention_rate,
"retention_note": (
"口径:登录(last_login_at)+开始比价(real_compare_start)+"
"开始领券(real_coupon_start/claim_started),按用户去重"
),
},
"comparison": {
"total": period_comparison_total,
"success": period_comparison_success,
"success_rate": period_comparison_success_rate,
"ordered": period_ordered_count,
"average_duration_ms": period_avg_duration_ms,
"average_saved_cents": period_avg_saved_cents,
},
"coins": {
"granted_total": _sum(CoinTransaction.amount, *period_coin_conds),
"reward_video_coin_total": period_reward_video_coin_total,
"feed_ad_coin_total": period_feed_ad_coin_total,
"signin_coin_total": period_signin_coin_total,
"signin_boost_coin_total": period_signin_boost_coin_total,
"task_coin_total": period_task_coin_total,
"coupon_reward_coin_total": period_coupon_reward_coin_total,
"comparison_reward_coin_total": period_comparison_reward_coin_total,
"regular_task_coin_total": period_regular_task_coin_total,
},
"cash": {
"withdraw_success_cents": _sum(
WithdrawOrder.amount_cents,
WithdrawOrder.status == "success",
WithdrawOrder.created_at >= start_local,
WithdrawOrder.created_at < end_local,
),
},
"trend": trend_points,
},
"feedback": {
"new": _count(Feedback, Feedback.status.in_(("pending", "new"))),
},
"cps": {
"available": True,
"note": "美团/JD CPS 读 cps_order 对账订单;淘宝佣金暂空",
"meituan_order_count": len(period_meituan_valid_orders),
"meituan_commission_cents": sum(
o.commission_cents or 0 for o in period_meituan_valid_orders
),
"meituan_hit_count": period_meituan_hit_count,
"meituan_miss_count": period_meituan_miss_count,
"meituan_unknown_rate_count": period_meituan_unknown_rate_count,
"meituan_hit_rate": period_meituan_hit_rate,
"jd_order_count": len(period_jd_valid_orders),
# 数据大盘京东 CPS 只看实际佣金,不再用预估佣金兜底。
"jd_commission_cents": sum(
o.actual_commission_cents or 0 for o in period_jd_valid_orders
),
"jd_actual_commission_cents": sum(
o.actual_commission_cents or 0 for o in period_jd_valid_orders
),
"jd_estimated_commission_cents": sum(
o.estimated_commission_cents or 0 for o in period_jd_valid_orders
),
"jd_invalid_count": len(period_jd_invalid_orders),
},
}
+11 -1
View File
@@ -56,6 +56,13 @@ def get_ad_revenue_report(
"全局筛选,同时影响明细 / 合计 / 趋势"
),
] = None,
app_env: Annotated[
str | None,
Query(
description="prod(正式应用) / test(测试应用);不传=全部。"
"建议正式收益报表选 prod,避免测试应用的假 eCPM 污染收益合计/平均"
),
] = None,
granularity: Annotated[
str, Query(description="day=按天 / hour=按小时(北京时间);区间>1 天建议用 day")
] = "day",
@@ -75,7 +82,7 @@ def get_ad_revenue_report(
result = ad_revenue.ad_revenue_report(
db, date_from=d_from.isoformat(), date_to=d_to.isoformat(),
user_id=user_id, ad_type=ad_type, feed_scene=feed_scene,
user_id=user_id, ad_type=ad_type, feed_scene=feed_scene, app_env=app_env,
granularity=granularity, limit=limit, offset=offset, sort=sort,
)
return AdRevenueReportOut(
@@ -89,6 +96,9 @@ def get_ad_revenue_report(
truncated=result["truncated"],
total_impressions=result["total_impressions"],
total_revenue_yuan=result["total_revenue_yuan"],
total_pangle_revenue_yuan=result["total_pangle_revenue_yuan"],
total_pangle_api_revenue_yuan=result["total_pangle_api_revenue_yuan"],
pangle_revenue_available=result["pangle_revenue_available"],
total_expected_coin=result["total_expected_coin"],
total_actual_coin=result["total_actual_coin"],
mismatch_count=result["mismatch_count"],
+106
View File
@@ -0,0 +1,106 @@
"""admin「领券数据」看板:发起/完成数 + 领券耗时(均值 + P5/P50/P95/P99)+ 按天趋势 + 逐条明细。
任意已登录 admin 可看(只读)聚合逻辑在 app/admin/repositories/coupon_data.py
数据源 coupon_session(客户端 /api/v1/coupon/session 两段上报)
"""
from __future__ import annotations
from datetime import date as _date
from typing import Annotated
from fastapi import APIRouter, Depends, HTTPException, Query
from app.admin.deps import AdminDb, get_current_admin
from app.admin.repositories import coupon_data
from app.admin.schemas.coupon_data import (
CouponDataDaily,
CouponDataHourly,
CouponDataOut,
CouponDataRow,
CouponDataSummary,
CouponUserRecordsOut,
)
from app.core.rewards import cn_today
router = APIRouter(
prefix="/admin/api/coupon-data",
tags=["admin-coupon-data"],
dependencies=[Depends(get_current_admin)],
)
# 区间最大跨度(天);超出拒绝,避免一次拉过多天拖垮接口(对齐广告收益报表)。
_MAX_RANGE_DAYS = 92
def _parse_day(value: str | None, *, field: str, default: _date) -> _date:
if value is None:
return default
try:
return _date.fromisoformat(value)
except ValueError as e:
raise HTTPException(status_code=422, detail=f"{field} 需为 YYYY-MM-DD") from e
@router.get(
"",
response_model=CouponDataOut,
summary="领券数据看板(发起/完成数 + 耗时分位 + 按天趋势 + 逐条明细)",
)
def get_coupon_data(
db: AdminDb,
date_from: Annotated[str | None, Query(description="起始日 北京 YYYY-MM-DD,默认今天")] = None,
date_to: Annotated[str | None, Query(description="结束日 北京 YYYY-MM-DD,闭区间,默认=date_from")] = None,
user: Annotated[str | None, Query(description="用户手机号/昵称模糊搜;不传=全部")] = None,
app_env: Annotated[str, Query(description="prod(默认) / dev / all(全部环境)")] = "prod",
granularity: Annotated[
str, Query(description="day=按天 / hour=按小时(北京);区间>1 天建议 day")
] = "day",
limit: Annotated[int, Query(ge=1, le=1000, description="每页条数(分页大小)")] = 500,
offset: Annotated[int, Query(ge=0, description="分页偏移(已跳过条数)=(页码-1)×每页条数")] = 0,
sort: Annotated[
str, Query(description="排序:time=发起时间倒序(默认) / elapsed=耗时倒序")
] = "time",
) -> CouponDataOut:
today = cn_today()
d_from = _parse_day(date_from, field="date_from", default=today)
d_to = _parse_day(date_to, field="date_to", default=d_from)
if d_to < d_from:
raise HTTPException(status_code=422, detail="date_to 不能早于 date_from")
if (d_to - d_from).days + 1 > _MAX_RANGE_DAYS:
raise HTTPException(status_code=422, detail=f"区间最长 {_MAX_RANGE_DAYS}")
# 报表默认只看 prod(对齐广告报表防串台口径);app_env=all 时不过滤、看全部环境。
env = None if app_env == "all" else app_env
result = coupon_data.coupon_data_report(
db, date_from=d_from.isoformat(), date_to=d_to.isoformat(),
user=user, app_env=env, granularity=granularity,
limit=limit, offset=offset, sort=sort,
)
return CouponDataOut(
date_from=d_from.isoformat(),
date_to=d_to.isoformat(),
summary=CouponDataSummary(**result["summary"]),
daily=[CouponDataDaily(**d) for d in result["daily"]],
hourly=[CouponDataHourly(**h) for h in result["hourly"]],
total=result["total"],
items=[CouponDataRow(**r) for r in result["items"]],
)
@router.get(
"/user-records",
response_model=CouponUserRecordsOut,
summary="某用户全部领券记录(点手机号抽屉:领券次数 + 记录列表)",
)
def get_user_coupon_records(
db: AdminDb,
user_id: Annotated[int, Query(description="用户 id")],
limit: Annotated[int, Query(ge=1, le=500, description="最多返回条数")] = 100,
sort_by: Annotated[str, Query(description="兼容 UserRecordsDrawer 参数;固定按发起时间倒序")] = "created_at",
sort_order: Annotated[str, Query(description="兼容参数,忽略")] = "desc",
) -> CouponUserRecordsOut:
result = coupon_data.coupon_user_records(db, user_id=user_id, limit=limit)
return CouponUserRecordsOut(
items=[CouponDataRow(**r) for r in result["items"]],
total=result["total"],
)
+95 -10
View File
@@ -1,13 +1,13 @@
"""admin CPS 分发与对账:群/活动管理 + 生成落地页短链 + 美团订单对账 + 统计。
"""admin CPS 分发与对账:群/活动管理 + 生成落地页短链 + 联盟订单对账 + 统计。
平台:meituan(actId+sid 转链 + query_order 对账) / taobao(整段淘口令) / jd(链接)
淘宝/京东无 API 只统计点击(咱落地页 PV/UV + 淘宝复制),对账字段显示 "-"
淘宝暂未接 API 只统计点击(咱落地页 PV/UV + 淘宝复制),对账字段显示 "-"
/活动管理 = operator;订单对账(涉佣金) = finance;只读列表/统计 = 登录即可
"""
from __future__ import annotations
import time
from datetime import datetime, timedelta, timezone
from datetime import date as _date, datetime, time as _dt_time, timedelta, timezone
from typing import Annotated
from fastapi import APIRouter, Depends, File, HTTPException, Query, Request, UploadFile
@@ -34,6 +34,7 @@ from app.admin.schemas.cps import (
from app.core import media
from app.core.config import settings
from app.integrations import meituan
from app.integrations.jd_union import JdUnionError
from app.integrations.meituan import MeituanCpsError
from app.models.admin import AdminUser
from app.models.cps_activity import CpsActivity
@@ -340,24 +341,108 @@ def generate_referral_links(
# ───────────── 订单对账 ─────────────
@router.post("/orders/reconcile", response_model=CpsReconcileResult, summary="拉取美团订单对账")
_BEIJING = timezone(timedelta(hours=8))
def _parse_day(value: str | None, *, field: str) -> _date | None:
if value is None:
return None
try:
return _date.fromisoformat(value)
except ValueError as e:
raise HTTPException(status_code=422, detail=f"{field} 需为 YYYY-MM-DD") from e
def _reconcile_range_to_ts(
date_from: _date | None, date_to: _date | None, days: int
) -> tuple[int, int]:
if date_from is None and date_to is None:
now = int(time.time())
return now - days * 86400, now
start_day = date_from or date_to
end_day = date_to or date_from
if start_day is None or end_day is None:
raise HTTPException(status_code=422, detail="日期参数不完整")
if start_day > end_day:
start_day, end_day = end_day, start_day
if (end_day - start_day).days + 1 > 90:
raise HTTPException(status_code=422, detail="美团订单查询最长 90 天")
start_dt = datetime.combine(start_day, _dt_time.min, tzinfo=_BEIJING)
end_dt = datetime.combine(end_day + timedelta(days=1), _dt_time.min, tzinfo=_BEIJING)
return int(start_dt.timestamp()), int(end_dt.timestamp())
def _reconcile_range_to_bj_dt(
date_from: _date | None, date_to: _date | None, days: int
) -> tuple[datetime, datetime]:
start_ts, end_ts = _reconcile_range_to_ts(date_from, date_to, days)
return (
datetime.fromtimestamp(start_ts, tz=_BEIJING),
datetime.fromtimestamp(end_ts, tz=_BEIJING),
)
def _merge_reconcile_result(total: dict, current: dict) -> None:
for key in ("fetched", "inserted", "updated", "pages"):
total[key] = int(total.get(key, 0)) + int(current.get(key, 0))
@router.post("/orders/reconcile", response_model=CpsReconcileResult, summary="拉取联盟订单对账")
def reconcile_orders(
request: Request,
admin: Annotated[AdminUser, Depends(require_role("finance"))],
db: AdminDb,
days: Annotated[int, Query(ge=1, le=90)] = 7,
date_from: Annotated[str | None, Query(description="起始日 YYYY-MM-DD")] = None,
date_to: Annotated[str | None, Query(description="结束日 YYYY-MM-DD")] = None,
days: Annotated[int, Query(ge=1, le=90, description="未传日期时默认拉近 N 天")] = 7,
sid: Annotated[str | None, Query(max_length=64)] = None,
query_time_type: Annotated[int, Query(ge=1, le=3)] = 1,
platform: Annotated[str, Query(pattern="^(all|meituan|jd)$")] = "all",
) -> CpsReconcileResult:
now = int(time.time())
parsed_from = _parse_day(date_from, field="date_from")
parsed_to = _parse_day(date_to, field="date_to")
result = {"fetched": 0, "inserted": 0, "updated": 0, "pages": 0}
try:
result = cps_repo.reconcile_orders(
db, start_time=now - days * 86400, end_time=now, sid=sid,
)
if platform in {"all", "meituan"}:
start_ts, end_ts = _reconcile_range_to_ts(parsed_from, parsed_to, days)
mt_result = cps_repo.reconcile_orders(
db,
start_time=start_ts,
end_time=end_ts,
query_time_type=query_time_type if query_time_type in (1, 2) else 2,
sid=sid,
)
_merge_reconcile_result(result, mt_result)
if platform in {"all", "jd"}:
if sid:
raise HTTPException(status_code=422, detail="京东订单刷新不支持 sid 筛选")
start_dt, end_dt = _reconcile_range_to_bj_dt(parsed_from, parsed_to, days)
jd_result = cps_repo.reconcile_jd_orders(
db,
start_time=start_dt,
end_time=end_dt,
query_time_type=query_time_type,
)
_merge_reconcile_result(result, jd_result)
except MeituanCpsError as e:
raise HTTPException(status_code=502, detail=f"美团拉单失败: {e}") from e
except JdUnionError as e:
raise HTTPException(status_code=502, detail=f"京东拉单失败: {e}") from e
write_audit(
db, admin, action="cps.order.reconcile", target_type="cps_order", target_id=None,
detail={"days": days, "sid": sid, **result}, ip=get_client_ip(request), commit=True,
detail={
"platform": platform,
"date_from": date_from,
"date_to": date_to,
"days": days,
"sid": sid,
"query_time_type": query_time_type,
**result,
},
ip=get_client_ip(request),
commit=True,
)
return CpsReconcileResult(**result)
+11 -3
View File
@@ -1,7 +1,9 @@
"""admin 数据大盘(只读聚合)。"""
from __future__ import annotations
from fastapi import APIRouter, Depends
from datetime import date
from fastapi import APIRouter, Depends, Query
from app.admin.deps import AdminDb, get_current_admin
from app.admin.repositories import stats
@@ -15,5 +17,11 @@ router = APIRouter(
@router.get("/overview", response_model=DashboardOverview, summary="大盘核心指标")
def overview(db: AdminDb) -> DashboardOverview:
return DashboardOverview.model_validate(stats.dashboard_overview(db))
def overview(
db: AdminDb,
date_from: date | None = Query(None, description="北京时间自然日起始日 YYYY-MM-DD"),
date_to: date | None = Query(None, description="北京时间自然日结束日 YYYY-MM-DD"),
) -> DashboardOverview:
return DashboardOverview.model_validate(
stats.dashboard_overview(db, date_from=date_from, date_to=date_to)
)
+11 -1
View File
@@ -10,7 +10,12 @@ from app.admin.audit import write_audit
from app.admin.deps import AdminDb, get_client_ip, get_current_admin, require_role
from app.admin.repositories import mutations, queries
from app.admin.schemas.common import CursorPage, OkResponse
from app.admin.schemas.feedback import FeedbackApproveRequest, FeedbackOut, FeedbackRejectRequest
from app.admin.schemas.feedback import (
FeedbackApproveRequest,
FeedbackOut,
FeedbackRejectRequest,
FeedbackSummary,
)
from app.models.admin import AdminUser
from app.models.feedback import Feedback
from app.repositories import wallet as wallet_repo
@@ -59,6 +64,11 @@ def list_feedbacks(
)
@router.get("/summary", response_model=FeedbackSummary, summary="反馈审核统计(各状态计数)")
def feedback_summary(db: AdminDb) -> FeedbackSummary:
return FeedbackSummary.model_validate(queries.feedback_summary(db))
@router.post("/{feedback_id}/handle", response_model=OkResponse, summary="标记反馈已处理")
def handle_feedback(
feedback_id: int,
+4 -1
View File
@@ -37,11 +37,14 @@ def list_price_reports(
db: AdminDb,
status: Annotated[str | None, Query()] = None,
user_id: Annotated[int | None, Query()] = None,
sort_by: Annotated[str, Query(pattern="^(id|created_at)$")] = "id",
sort_order: Annotated[str, Query(pattern="^(asc|desc)$")] = "desc",
limit: Annotated[int, Query(ge=1, le=100)] = 20,
cursor: Annotated[int | None, Query()] = None,
) -> CursorPage[PriceReportOut]:
items, next_cursor, total = queries.list_price_reports(
db, status=status, user_id=user_id, limit=limit, cursor=cursor,
db, status=status, user_id=user_id,
sort_by=sort_by, sort_order=sort_order, limit=limit, cursor=cursor,
)
return CursorPage(
items=[PriceReportOut.model_validate(r) for r in items],
+19 -11
View File
@@ -210,22 +210,28 @@ def grant_user_cash(
db: AdminDb,
) -> OkResponse:
"""给指定用户增/减或设值现金(分)。delta:正=发放、负=扣减;set:直接设为目标值。
account=coin_cash(金币兑现金)/ invite_cash(邀请奖励金):两本账物理隔离各调各的
主要用于让无现金用户直接测试提现"""
user = user_repo.get_user_by_id(db, user_id)
if user is None:
raise HTTPException(status_code=404, detail="用户不存在")
# set=设为目标值:读当前余额算差值,仍复用 grant_cash 写一笔流水(沿用原子/审计/扣负保护)
# 按目标账户选「余额字段 + 变动入口」(grant_invite_cash 与 grant_cash 同构)
is_invite = body.account == "invite_cash"
balance_attr = "invite_cash_balance_cents" if is_invite else "cash_balance_cents"
grant_fn = wallet_repo.grant_invite_cash if is_invite else wallet_repo.grant_cash
acct_label = "邀请奖励金" if is_invite else "现金"
before: int | None = None
# set=设为目标值:读当前余额算差值,仍复用 grant 写一笔流水(沿用原子/审计/扣负保护)
if body.mode == "set":
if body.amount_cents < 0:
raise HTTPException(status_code=400, detail="目标现金值不能为负")
raise HTTPException(status_code=400, detail=f"目标{acct_label}值不能为负")
# lock=True:锁账户行,防连点/并发各读同一 before 算同一 delta 双写,余额错位
before = wallet_repo.get_or_create_account(
db, user_id, commit=False, lock=True
).cash_balance_cents
acc_locked = wallet_repo.get_or_create_account(db, user_id, commit=False, lock=True)
before = getattr(acc_locked, balance_attr)
delta = body.amount_cents - before
if delta == 0:
raise HTTPException(
status_code=400, detail=f"当前现金已为 {body.amount_cents} 分,无需调整"
status_code=400, detail=f"当前{acct_label}已为 {body.amount_cents} 分,无需调整"
)
else:
if body.amount_cents == 0:
@@ -234,18 +240,20 @@ def grant_user_cash(
# 负数扣减时不允许扣成负余额(运营误操作保护);lock=True 防并发扣穿
if delta < 0:
acc_now = wallet_repo.get_or_create_account(db, user_id, commit=False, lock=True)
if acc_now.cash_balance_cents + delta < 0:
if getattr(acc_now, balance_attr) + delta < 0:
raise HTTPException(
status_code=400, detail=f"扣减后现金为负(当前余额 {acc_now.cash_balance_cents} 分)"
status_code=400,
detail=f"扣减后{acct_label}为负(当前余额 {getattr(acc_now, balance_attr)} 分)",
)
biz_type = "admin_grant" if delta > 0 else "admin_deduct"
# grant_cash 只 flush 不 commit;审计同 commit=False;最后一起 commit → 原子(改钱+留痕)
acc, _ = wallet_repo.grant_cash(
# grant 只 flush 不 commit;审计同 commit=False;最后一起 commit → 原子(改钱+留痕)
acc, _ = grant_fn(
db, user_id, delta, biz_type=biz_type, remark=f"admin:{body.reason}"[:128],
)
detail = {
"account": body.account,
"amount_cents": delta,
"balance_after_cents": acc.cash_balance_cents,
"balance_after_cents": getattr(acc, balance_attr),
"reason": body.reason,
}
if body.mode == "set":
+36 -2
View File
@@ -44,7 +44,13 @@ class AdRevenueDaily(BaseModel):
date: str = Field(..., description="北京时间 YYYY-MM-DD")
impressions: int = Field(..., description="当天展示条数合计")
revenue_yuan: float = Field(..., description="当天预估收益合计(元)")
revenue_yuan: float = Field(..., description="当天客户端预估收益合计(元;eCPM 折算)")
pangle_revenue_yuan: float | None = Field(
None, description="当天穿山甲后台预估收益(元;GroMore revenue);非全量视图/无数据为空"
)
pangle_api_revenue_yuan: float | None = Field(
None, description="当天穿山甲收益Api(元;GroMore api_revenue,更接近结算);未配/当天/无数据为空"
)
expected_coin: int = Field(..., description="当天应发金币合计")
actual_coin: int = Field(..., description="当天实发金币合计")
@@ -88,6 +94,11 @@ class AdRevenueRow(BaseModel):
impressions: int = Field(..., description="本行展示条数:有展示=1 / 纯发奖=0(供日汇总、趋势图复用)")
ecpm: str | None = Field(None, description="eCPM 原始值(分/千次);展示行取展示值,纯发奖行取发奖采用值")
revenue_yuan: float = Field(..., description="本次展示预估收益(元)= eCPM元 ÷ 1000;纯发奖行=0")
row_revenue_yuan: float | None = Field(
None,
description="主表逐行展示用的预估收益(元):一次比价/领券聚合行=该次发奖广告 eCPM 折算之和;"
"其它行为空(前端回退取 revenue_yuan)。不进合计/趋势,避免与展示侧重复计",
)
adn: str | None = Field(None, description="实际填充 ADN 子渠道(pangle/gdt…);纯发奖行为空")
slot_id: str | None = Field(None, description="底层 mediation rit(非我们配置的广告位 ID);纯发奖行为空")
# ── 发奖侧 ──
@@ -100,6 +111,15 @@ class AdRevenueRow(BaseModel):
None,
description="发奖复算明细(eCPM/因子1/份数/LT/因子2/应发/实发/一致);点行展开下钻用,纯展示为空",
)
sub_rewards: list[AdRevenueRecord] = Field(
default_factory=list,
description="一次比价/领券聚合行的组内逐条发奖明细(同一整场 ad_session_id 的多条广告);"
"点行展开渲染多行。激励视频/纯展示行为空(单条看 reward_detail)",
)
sub_count: int = Field(
1,
description="本行聚合的发奖条数:一次比价/领券=该次广告条数(≥1);激励视频/纯展示=1",
)
class AdRevenueReportOut(BaseModel):
@@ -123,7 +143,21 @@ class AdRevenueReportOut(BaseModel):
total: int = Field(..., description="广告事件总数(全量,不受分页影响;= 当前筛选下的分页总条数)")
truncated: bool = Field(..., description="当前页之后是否还有更多事件(len(events) > offset + limit)")
total_impressions: int = Field(..., description="全量展示条数合计")
total_revenue_yuan: float = Field(..., description="全量收益合计(元)")
total_revenue_yuan: float = Field(..., description="全量客户端预估收益合计(元;eCPM 折算)")
total_pangle_revenue_yuan: float | None = Field(
None,
description="全量穿山甲后台预估收益合计(元;GroMore revenue)。穿山甲无用户/类型/场景维度,"
"仅「全量视图」(未按 user_id/ad_type/feed_scene 过滤)时有值,否则为 null",
)
total_pangle_api_revenue_yuan: float | None = Field(
None,
description="全量穿山甲收益Api合计(元;GroMore api_revenue,各 ADN 回传、更接近结算);"
"未配 Reporting / 查当天 / 非全量视图 时为 null",
)
pangle_revenue_available: bool = Field(
False,
description="本次结果是否带穿山甲后台收益(=全量视图且已同步到数据)。false 时前端「穿山甲收益」显示「-」",
)
total_expected_coin: int = Field(..., description="全量应发金币合计")
total_actual_coin: int = Field(..., description="全量实发金币合计")
mismatch_count: int = Field(..., description="应发≠实发的发奖条数(=0 说明全部按公式发放)")
+83
View File
@@ -0,0 +1,83 @@
"""admin「领券数据」看板 schemas:汇总卡 + 按天/小时趋势 + 逐条领券明细。
数据源 coupon_session(一次领券一行)耗时单位 ms(前端按需折秒);均值/分位只统计 completed
"""
from __future__ import annotations
from datetime import datetime
from pydantic import BaseModel, Field
class CouponDataSummary(BaseModel):
"""汇总卡:发起/完成数 + 耗时均值与分位(P5/P50/P95/P99,基于 completed 的 elapsed_ms)。"""
started_count: int = Field(..., description="发起数(区间内所有领券 session)")
completed_count: int = Field(..., description="完成数(status=completed)")
avg_elapsed_ms: int | None = Field(None, description="平均耗时(ms,仅 completed;无数据为空)")
p5_ms: int | None = Field(None, description="耗时 5 分位(ms)")
p50_ms: int | None = Field(None, description="耗时 50 分位(ms,中位数)")
p95_ms: int | None = Field(None, description="耗时 95 分位(ms)")
p99_ms: int | None = Field(None, description="耗时 99 分位(ms)")
class CouponDataDaily(BaseModel):
"""按天趋势(全量,不受分页影响):柱=发起/完成数,线=平均耗时。"""
date: str = Field(..., description="北京时间 YYYY-MM-DD")
started_count: int
completed_count: int
avg_elapsed_ms: int | None = Field(None, description="当天平均耗时(ms,仅 completed)")
class CouponDataHourly(BaseModel):
"""按北京小时(023)趋势(单日 granularity=hour 时非空)。"""
hour: int = Field(..., description="北京时间小时 023")
started_count: int
completed_count: int
avg_elapsed_ms: int | None = None
class CouponDataRow(BaseModel):
"""一条领券明细(一次领券任务)。"""
id: int = Field(..., description="coupon_session 主键(抽屉 rowKey 用)")
trace_id: str
user_id: int | None = None
user_phone: str | None = Field(None, description="手机号(admin 展示;匿名领券/查不到为空)")
user_nickname: str | None = Field(None, description="昵称")
status: str = Field(..., description="started / completed / failed / abandoned")
platforms: list[str] | None = Field(None, description="发起勾选平台")
origin_package: str | None = Field(None, description="发起来源 App 包名;null=App 内(傻瓜比价首页)发起")
elapsed_ms: int | None = Field(None, description="全程耗时(ms)")
platform_elapsed: dict[str, int] | None = Field(
None, description="各平台耗时 {meituan-waimai/taobao-shanguang/jd-waimai: ms}"
)
device_model: str | None = None
rom: str | None = None
app_env: str | None = None
started_at: datetime = Field(..., description="发起时刻(明细「时间」列)")
claimed_count: int | None = None
trace_url: str | None = Field(None, description="pricebot 公网 trace 链接(仅 completed 有);admin 渲染可点链接,无则显示可复制 trace_id")
class CouponDataOut(BaseModel):
"""领券数据看板响应:汇总卡 + 趋势 + 明细分页。"""
date_from: str
date_to: str
summary: CouponDataSummary
daily: list[CouponDataDaily] = Field(default_factory=list, description="按天趋势(全量)")
hourly: list[CouponDataHourly] = Field(
default_factory=list, description="按小时趋势(单日 hour 粒度时非空)"
)
total: int = Field(..., description="明细总条数(全量,不受分页)")
items: list[CouponDataRow] = Field(..., description="逐条领券明细(当前页)")
class CouponUserRecordsOut(BaseModel):
"""某用户全部领券记录(点手机号抽屉用):total=该用户领券总次数,items=记录列表(UserRecordsDrawer 渲染)。"""
items: list[CouponDataRow]
total: int
+9 -2
View File
@@ -1,7 +1,7 @@
"""admin CPS 分发与对账 schemas。金额统一「分」(cents),前端 yuan() 展示。
平台:meituan(actId+sid 转链对账) / taobao(淘口令,只统计点击) / jd(链接,只统计点击)
对账类字段对淘宝/京东 None 前端显示 "-"(无法对账)
平台:meituan(actId+sid 转链对账) / taobao(淘口令,只统计点击) / jd(链接 + 订单 API 对账)
对账类字段对淘宝为 None 前端显示 "-"(暂未对账)
"""
from __future__ import annotations
@@ -116,15 +116,22 @@ class CpsOrderOut(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: int
platform: str = "meituan"
order_id: str
external_order_id: str | None = None
external_row_id: str | None = None
sid: str | None = None
act_id: str | None = None
pay_price_cents: int | None = None
commission_cents: int | None = None
estimated_commission_cents: int | None = None
actual_commission_cents: int | None = None
commission_rate: str | None = None
mt_status: str | None = None
jd_valid_code: str | None = None
invalid_reason: str | None = None
product_name: str | None = None
settle_month: str | None = None
pay_time: datetime | None = None
+64
View File
@@ -1,6 +1,8 @@
"""admin 大盘 schemas(对应 stats.dashboard_overview 的嵌套结构)。"""
from __future__ import annotations
from datetime import date
from pydantic import BaseModel
@@ -38,6 +40,56 @@ class DashboardComparison(BaseModel):
success_rate: float
class DashboardPeriodUsers(BaseModel):
new: int
active: int
retained_new_users: int
retention_rate: float | None = None
retention_note: str
class DashboardPeriodComparison(BaseModel):
total: int
success: int
success_rate: float
ordered: int
average_duration_ms: int | None = None
average_saved_cents: int | None = None
class DashboardPeriodCoins(BaseModel):
granted_total: int
reward_video_coin_total: int = 0
feed_ad_coin_total: int = 0
signin_coin_total: int = 0
signin_boost_coin_total: int = 0
task_coin_total: int = 0
coupon_reward_coin_total: int = 0
comparison_reward_coin_total: int = 0
regular_task_coin_total: int = 0
class DashboardPeriodCash(BaseModel):
withdraw_success_cents: int
class DashboardTrendPoint(BaseModel):
date: date
active_users: int
new_users: int
comparisons: int
class DashboardPeriod(BaseModel):
date_from: date
date_to: date
users: DashboardPeriodUsers
comparison: DashboardPeriodComparison
coins: DashboardPeriodCoins
cash: DashboardPeriodCash
trend: list[DashboardTrendPoint] = []
class DashboardFeedback(BaseModel):
new: int
@@ -45,6 +97,17 @@ class DashboardFeedback(BaseModel):
class DashboardCps(BaseModel):
available: bool
note: str
meituan_order_count: int = 0
meituan_commission_cents: int = 0
meituan_hit_count: int = 0
meituan_miss_count: int = 0
meituan_unknown_rate_count: int = 0
meituan_hit_rate: float | None = None
jd_order_count: int = 0
jd_commission_cents: int = 0
jd_actual_commission_cents: int = 0
jd_estimated_commission_cents: int = 0
jd_invalid_count: int = 0
class DashboardOverview(BaseModel):
@@ -52,5 +115,6 @@ class DashboardOverview(BaseModel):
coins: DashboardCoins
cash: DashboardCash
comparison: DashboardComparison
period: DashboardPeriod
feedback: DashboardFeedback
cps: DashboardCps
+17
View File
@@ -23,6 +23,14 @@ class FeedbackOut(BaseModel):
reviewed_by_admin_id: int | None = None
reviewed_at: datetime | None = None
created_at: datetime
# 提交端环境快照(feedback 表列):提交版本号 / 机型OS版本;改版前的历史反馈为 None
app_version: str | None = None
device_model: str | None = None
rom_name: str | None = None
android_version: str | None = None
# 联表瞬态字段(queries._attach_user_info 挂):列表展示完整手机号,点手机号查该用户全部反馈
phone: str | None = None
nickname: str | None = None
class FeedbackApproveRequest(BaseModel):
@@ -37,3 +45,12 @@ class FeedbackApproveRequest(BaseModel):
class FeedbackRejectRequest(BaseModel):
reason: str = Field(min_length=1, max_length=256, description="未采纳原因,用户端可见")
note: str | None = Field(default=None, max_length=256, description="运营内部审核备注")
class FeedbackSummary(BaseModel):
"""审核台顶部各状态计数(pending 含历史 new 态)。"""
pending: int
adopted: int
rejected: int
total: int
+11
View File
@@ -31,6 +31,17 @@ class PriceReportOut(BaseModel):
reward_coins: int | None = None
reviewed_at: datetime | None = None
created_at: datetime
# 联表瞬态字段:phone/nickname 由 _attach_user_info 挂(展示完整手机号、点击查该用户全部上报);
# 其余由 _attach_price_report_comparison 从关联比价记录挂:trace_*(Trace 列点跳调试链接)、
# device_model/rom_name/android_version(机型OS版本列)、app_version(提交版本号列)
phone: str | None = None
nickname: str | None = None
trace_id: str | None = None
trace_url: str | None = None
device_model: str | None = None
rom_name: str | None = None
android_version: str | None = None
app_version: str | None = None
class PriceReportRejectRequest(BaseModel):
+7
View File
@@ -17,6 +17,7 @@ class AdminUserListItem(BaseModel):
status: str
debug_trace_enabled: bool = False
wechat_openid: str | None = None
wechat_nickname: str | None = None
created_at: datetime
last_login_at: datetime
@@ -29,6 +30,7 @@ class AdminUserOverview(BaseModel):
user: AdminUserListItem
coin_balance: int
cash_balance_cents: int
invite_cash_balance_cents: int # 邀请奖励金余额(与 cash_balance_cents 物理隔离)
total_coin_earned: int
comparison_total: int
comparison_success: int
@@ -88,6 +90,11 @@ class GrantCoinsRequest(BaseModel):
class GrantCashRequest(BaseModel):
# 目标账户:金币兑换的现金(cash_balance_cents)与邀请奖励金(invite_cash_balance_cents)物理隔离,
# 各调各的、不可串。默认 coin_cash 兼容旧调用。
account: Literal["coin_cash", "invite_cash"] = Field(
"coin_cash", description="目标账户:coin_cash=金币兑现金账户 / invite_cash=邀请奖励金账户"
)
mode: Literal["delta", "set"] = Field(
"delta", description="delta=增减(amount_cents 为变动量) / set=设为(amount_cents 为目标值,须≥0)"
)
+33
View File
@@ -11,6 +11,7 @@ from __future__ import annotations
import hmac
import logging
from datetime import datetime, timedelta, timezone
from typing import Annotated
from fastapi import APIRouter, Header, HTTPException, status
@@ -21,6 +22,7 @@ from app.repositories import launch_confirm_sample as repo
from app.schemas.launch_confirm_sample import (
LaunchConfirmSampleIn,
LaunchConfirmSampleOut,
LaunchConfirmSampleRow,
)
logger = logging.getLogger("shagua.internal.launch_confirm")
@@ -61,3 +63,34 @@ def report_launch_confirm_sample(
payload.exec_success, payload.trace_id,
)
return LaunchConfirmSampleOut(id=sid)
@router.get(
"/launch-confirm-samples",
response_model=list[LaunchConfirmSampleRow],
summary="启动确认窗兜底样本列表(沉淀脚本 distill 读; server→server, 不走 JWT)",
)
def list_launch_confirm_samples(
db: DbSession,
x_internal_secret: Annotated[str | None, Header()] = None,
exec_success: bool | None = None,
host_package: str | None = None,
since_days: int | None = None,
limit: int = 1000,
) -> list[LaunchConfirmSampleRow]:
"""读样本供 pricebot 的 distill_launch_confirm.py 聚合沉淀回 PROFILES。
与上报端点同一把共享密钥;exec_success/host_package/since_days 均可选,limit 默认 1000
"""
_check_secret(x_internal_secret)
since = None
if since_days and since_days > 0:
since = datetime.now(timezone.utc) - timedelta(days=since_days)
rows = repo.list_samples(
db,
exec_success=exec_success,
host_package=host_package,
since=since,
limit=limit,
)
return [LaunchConfirmSampleRow.model_validate(r) for r in rows]
+5 -4
View File
@@ -26,6 +26,7 @@ import httpx
from fastapi import APIRouter, HTTPException, Request, status
from app.core.config import settings
from app.core.pricebot_client import get_pricebot_client
from app.core.pricebot_router import pick_pricebot
logger = logging.getLogger("shagua.compare")
@@ -65,10 +66,10 @@ async def _passthrough(request: Request, upstream_path: str) -> dict[str, Any]:
)
try:
async with httpx.AsyncClient(timeout=timeout) as client:
resp = await client.post(
url, content=raw, headers={"Content-Type": "application/json"}
)
client = get_pricebot_client()
resp = await client.post(
url, content=raw, headers={"Content-Type": "application/json"}, timeout=timeout
)
except httpx.RequestError as e:
logger.error("[pricebot] request failed: %s", e)
raise HTTPException(
+34 -4
View File
@@ -20,6 +20,7 @@ from fastapi.concurrency import run_in_threadpool
from app.api.deps import CurrentUser, DbSession
from app.core.config import settings
from app.core.pricebot_client import get_pricebot_client
from app.core.pricebot_router import pick_pricebot
from app.db.session import SessionLocal
from app.repositories import coupon_state as coupon_repo
@@ -28,6 +29,7 @@ from app.schemas.coupon_state import (
CouponPromptDismissIn,
CouponPromptShouldShowOut,
CouponPromptShownIn,
CouponSessionIn,
CouponStatsOut,
)
@@ -141,10 +143,10 @@ async def coupon_step(
)
try:
async with httpx.AsyncClient(timeout=timeout) as client:
resp = await client.post(
url, content=raw, headers={"Content-Type": "application/json"}
)
client = get_pricebot_client()
resp = await client.post(
url, content=raw, headers={"Content-Type": "application/json"}, timeout=timeout
)
except httpx.RequestError as e:
logger.error("[pricebot] request failed: %s", e)
raise HTTPException(
@@ -193,6 +195,34 @@ async def coupon_step(
return resp_json
@router.post("/session", summary="领券任务流水上报(admin 领券数据看板数据源)")
def coupon_session(payload: CouponSessionIn, db: DbSession) -> dict[str, bool]:
"""客户端两段上报一次领券流水(发起 started / 收尾 completed-failed-abandoned),按 trace_id upsert
coupon_session不鉴权(同领券循环 MVP, device_id/trace_id); admin领券数据看板算
发起/完成数耗时分位机型维度写库失败不应连累客户端(本就 fire-and-forget),吞掉返回 ok"""
try:
coupon_repo.upsert_coupon_session(
db,
trace_id=payload.trace_id,
device_id=payload.device_id,
status=payload.status,
started_at_ms=payload.started_at_ms,
user_id=payload.user_id,
platforms=payload.platforms,
origin_package=payload.origin_package,
device_model=payload.device_model,
rom=payload.rom,
app_env=payload.app_env,
elapsed_ms=payload.elapsed_ms,
platform_elapsed=payload.platform_elapsed,
claimed_count=payload.claimed_count,
trace_url=payload.trace_url,
)
except Exception as e: # noqa: BLE001
logger.warning("coupon session write failed: %s", e)
return {"ok": True}
@router.post("/prompt/shown", summary="领券引导窗弹出即上报(按 App 记 shown)")
def coupon_prompt_shown(payload: CouponPromptShownIn, db: DbSession) -> dict[str, bool]:
"""客户端弹出引导窗那刻调 → 记一条今日 engagement(shown),今天**这个 App** 不再自动弹。
+7
View File
@@ -61,6 +61,11 @@ async def submit_feedback(
content: str = Form(...),
# 原型改版后客户端不再采集联系方式;保留字段以兼容旧端 + 后续可能复用,默认空串。
contact: str = Form(default=""),
# 提交端环境快照(admin 反馈页展示「提交版本号」「机型OS版本」);旧端不带 → 空串 → 存 NULL。
app_version: str = Form(default=""),
device_model: str = Form(default=""),
rom_name: str = Form(default=""),
android_version: str = Form(default=""),
images: list[UploadFile] = File(default=[]),
) -> FeedbackOut:
content = content.strip()
@@ -86,6 +91,8 @@ async def submit_feedback(
fb = feedback_repo.create_feedback(
db, user_id=user.id, content=content, contact=contact, images=urls,
app_version=app_version.strip(), device_model=device_model.strip(),
rom_name=rom_name.strip(), android_version=android_version.strip(),
)
logger.info("feedback id=%d user_id=%d images=%d", fb.id, user.id, len(urls))
return FeedbackOut.model_validate(fb)
+3
View File
@@ -202,6 +202,9 @@ def withdraw(req: WithdrawRequest, user: CurrentUser, db: DbSession) -> Withdraw
order = crud_wallet.create_withdraw(
db, user.id, req.amount_cents, source=req.source,
user_name=req.user_name, out_bill_no=req.out_bill_no,
# 0.01 元调试提现:放行低于最低额的小额。双闸——客户端仅 debug 包在「0.01 元提现」开关开时
# 连同 skip_review 一起下发;服务端仅非 prod 才认。生产恒 False,最低额校验照常。
allow_sub_min=(req.skip_review and not settings.is_prod),
)
except crud_wallet.InvalidWithdrawAmountError as e:
raise HTTPException(
+50
View File
@@ -113,6 +113,21 @@ class Settings(BaseSettings):
"""美团 CPS 凭证齐全(缺则接口返空,而非 502)。"""
return bool(self.MT_CPS_APP_KEY and self.MT_CPS_APP_SECRET)
# ===== 京东联盟 CPS =====
# app_key/app_secret 来自京东联盟应用;site_id 是推广管理里的 APP/网站 ID;
# auth_key 是工具商授权 key,自有应用查询可留空。
JD_UNION_APP_KEY: str = ""
JD_UNION_APP_SECRET: str = ""
JD_UNION_SITE_ID: str = ""
JD_UNION_AUTH_KEY: str = ""
JD_UNION_GATEWAY: str = "https://api.jd.com/routerjson"
JD_UNION_TIMEOUT_SEC: int = 15
@property
def jd_union_configured(self) -> bool:
"""京东联盟订单查询凭证齐全。"""
return bool(self.JD_UNION_APP_KEY and self.JD_UNION_APP_SECRET)
# ===== 微信服务号(网页授权) =====
# CPS 落地页在微信内拿用户 openid(base 静默)/昵称头像(userinfo),做用户级群统计。
# ⚠️ 区别于 WECHAT_APP_ID(那是 App 移动应用,用于微信支付);这是【已认证服务号】。
@@ -211,6 +226,41 @@ class Settings(BaseSettings):
"""回调开关打开且至少配了一个验签密钥,才接受发奖回调。"""
return bool(self.PANGLE_CALLBACK_ENABLED and self.pangle_reward_secrets)
# ===== 穿山甲 GroMore 数据 API(报表收益拉取,T+1)=====
# ⚠️ 与上面发奖回调的 m-key 是【两套完全不同的凭证】:这三样在穿山甲后台
# 「接入中心 → GroMore-API → 聚合数据报告 API」文档页领取(user_id / role_id / Security Key),
# 仅用于按天拉 GroMore 收益报表(revenue 预估收益 + api_revenue 收益Api),不参与发奖。
# 该 API 只能查【GroMore 聚合代码位】的数据(=我们 useMediation 的口径),非穿山甲 SDK 数据;
# 且不提供用户/设备维度(官方明确),故收益只能落到 日期×代码位 汇总,不能挂到逐条事件。
# 子账号(role_id≠user_id)需主账号在「角色管理」授予「查看全部数据」权限,否则查不到
# ecpm/revenue(接口返回 118);role_id 填成与 user_id 一致 = 查主账号数据。
PANGLE_REPORT_USER_ID: int = 0 # 媒体账号 user_id
PANGLE_REPORT_ROLE_ID: int = 0 # 子账号 role_id(=user_id 时查主账号)
PANGLE_REPORT_SECURITY_KEY: str = "" # 该账号的 Security Key(secure_key,≠ 发奖 m-key)
# GroMore 聚合 AppId(报表 site_id 维度)→ 我们的应用环境。报表按 site_id 区分两个穿山甲应用,
# 用它把每行归到 prod(傻瓜比价正式)/ test(测试应用)。默认值取自现网两个应用,部署时按需覆盖。
PANGLE_REPORT_SITE_ID_PROD: str = "5830519" # 傻瓜比价正式应用 AppId
PANGLE_REPORT_SITE_ID_TEST: str = "5832303" # 测试应用 AppId
@property
def pangle_report_configured(self) -> bool:
"""三样齐全(user_id / role_id / security_key)才能拉 GroMore 报表;缺任一 → 同步脚本 no-op。"""
return bool(
self.PANGLE_REPORT_USER_ID
and self.PANGLE_REPORT_ROLE_ID
and self.PANGLE_REPORT_SECURITY_KEY
)
@property
def pangle_report_site_id_to_env(self) -> dict[str, str]:
"""site_id(穿山甲 AppId)→ app_env(prod/test);供同步脚本把报表行归到我们的应用。留空项忽略。"""
out: dict[str, str] = {}
if self.PANGLE_REPORT_SITE_ID_PROD.strip():
out[self.PANGLE_REPORT_SITE_ID_PROD.strip()] = "prod"
if self.PANGLE_REPORT_SITE_ID_TEST.strip():
out[self.PANGLE_REPORT_SITE_ID_TEST.strip()] = "test"
return out
# ===== Pricebot 上游 (领券/比价业务透传目标) =====
# pricebot-backend 默认跑在 8000。/api/v1/coupon/step 会透传到这里的 /api/coupon/step
PRICEBOT_BASE_URL: str = "http://localhost:8000"
+35
View File
@@ -0,0 +1,35 @@
"""透传到 pricebot 的共享 httpx.AsyncClient 单例。
为什么不能每请求新建(coupon.py / compare.py 老写法 async with httpx.AsyncClient(...)):
每次构造都重建一套 SSL 上下文(httpx.create_ssl_context 加载 certifi CA),实测
~1s+/; pricebot 是纯 http 透传,根本用不到 TLS 纯浪费,且每帧重交一次
trust_env 默认 True 会读进程 HTTP_PROXY, http://localhost:8000 这条本地透传整个
塞进本机代理( Clash 7897),恒定再多几秒
单例:启动只建一次(SSL/连接池一次性),keep-alive 复用 TCP,每帧降到个位数 ms
trust_env=False:对齐 integrations/meituan.py 的既有约定,不被进程代理误导,直连 pricebot
"""
from __future__ import annotations
import httpx
_client: httpx.AsyncClient | None = None
def get_pricebot_client() -> httpx.AsyncClient:
"""取透传单例。lifespan 启动会预热;未预热(如测试态)懒建兜底。
超时不在此固化(coupon 30s / compare 60s 不同),由调用点 client.post(timeout=...)
懒建无 await,asyncio 单线程下不会有并发竞态
"""
global _client
if _client is None:
_client = httpx.AsyncClient(trust_env=False)
return _client
async def aclose_pricebot_client() -> None:
"""lifespan 关停时调,优雅关连接池。"""
global _client
if _client is not None:
await _client.aclose()
_client = None
+163
View File
@@ -0,0 +1,163 @@
"""京东联盟 OpenAPI 客户端。
当前只接数据大盘需要的订单明细接口:
`jd.union.open.order.row.query`京东要求订单查询时间窗最长 1 小时,
调用方负责切窗分页
"""
from __future__ import annotations
import hashlib
import json
import logging
from datetime import datetime, timedelta, timezone
from typing import Any
import httpx
from app.core.config import settings
logger = logging.getLogger(__name__)
_BEIJING = timezone(timedelta(hours=8))
class JdUnionError(RuntimeError):
"""京东联盟 API 调用失败。"""
def _parse_json_maybe(value: Any) -> Any:
if not isinstance(value, str):
return value
text = value.strip()
if not text:
return value
try:
return json.loads(text)
except json.JSONDecodeError:
return value
def _sign(params: dict[str, Any], secret: str) -> str:
pieces = [secret]
for key in sorted(k for k in params if k != "sign"):
value = params[key]
if value is None:
continue
pieces.append(f"{key}{value}")
pieces.append(secret)
raw = "".join(pieces)
return hashlib.md5(raw.encode("utf-8")).hexdigest().upper()
def _unwrap_response(data: dict[str, Any]) -> dict[str, Any]:
if "error_response" in data:
err = data["error_response"] or {}
msg = err.get("zh_desc") or err.get("en_desc") or err.get("msg") or err
raise JdUnionError(f"京东 API 错误: {msg}")
body: Any = data
for key, value in data.items():
if key.endswith("_responce") or key.endswith("_response"):
body = value
break
body = _parse_json_maybe(body)
if not isinstance(body, dict):
raise JdUnionError("京东 API 返回格式异常")
result = body.get("queryResult", body.get("result", body))
result = _parse_json_maybe(result)
if not isinstance(result, dict):
raise JdUnionError("京东 API 业务结果格式异常")
code = str(result.get("code", result.get("resultCode", "200")))
if code not in {"0", "200"}:
msg = result.get("message") or result.get("msg") or result.get("resultMsg") or result
raise JdUnionError(f"京东 API 业务错误: {msg}")
return result
def call(method: str, payload: dict[str, Any], *, version: str = "1.0") -> dict[str, Any]:
if not settings.jd_union_configured:
raise JdUnionError("京东联盟凭证未配置")
biz_json = json.dumps(payload, ensure_ascii=False, separators=(",", ":"))
params: dict[str, Any] = {
"method": method,
"app_key": settings.JD_UNION_APP_KEY,
"timestamp": datetime.now(_BEIJING).strftime("%Y-%m-%d %H:%M:%S"),
"format": "json",
"v": version,
"sign_method": "md5",
"360buy_param_json": biz_json,
}
params["sign"] = _sign(params, settings.JD_UNION_APP_SECRET)
try:
with httpx.Client(timeout=settings.JD_UNION_TIMEOUT_SEC, trust_env=False) as client:
resp = client.post(settings.JD_UNION_GATEWAY, data=params)
resp.raise_for_status()
data = resp.json()
except httpx.HTTPError as e:
raise JdUnionError(f"京东 API 网络错误: {e}") from e
except json.JSONDecodeError as e:
raise JdUnionError("京东 API 返回非 JSON") from e
return _unwrap_response(data)
def _extract_rows(result: dict[str, Any]) -> tuple[list[dict[str, Any]], bool]:
payload = _parse_json_maybe(result.get("data", result.get("result", result)))
has_more = bool(result.get("hasMore") or result.get("has_more"))
if isinstance(payload, dict):
for key in ("orderRowResp", "orderRows", "orderList", "orders", "list", "rows"):
rows = _parse_json_maybe(payload.get(key))
if isinstance(rows, list):
return [r for r in rows if isinstance(r, dict)], bool(
payload.get("hasMore") or payload.get("has_more") or has_more
)
return [], bool(payload.get("hasMore") or payload.get("has_more") or has_more)
if isinstance(payload, list):
return [r for r in payload if isinstance(r, dict)], has_more
return [], has_more
def query_order_rows(
*,
start_time: datetime,
end_time: datetime,
query_time_type: int = 3,
page_index: int = 1,
page_size: int = 200,
) -> dict[str, Any]:
"""查询京东 CPS 订单行。
query_time_type: 1 下单时间, 2 完成时间, 3 更新时间
start_time/end_time 用北京时间展示给京东;调用方需保证窗口不超过 1 小时
"""
start_bj = start_time.astimezone(_BEIJING)
end_bj = end_time.astimezone(_BEIJING)
if end_bj <= start_bj:
return {"rows": [], "has_more": False}
if end_bj - start_bj > timedelta(hours=1):
raise JdUnionError("京东订单查询单次时间窗不能超过 1 小时")
order_req: dict[str, Any] = {
"pageIndex": page_index,
"pageSize": min(max(page_size, 1), 200),
"type": query_time_type,
"startTime": start_bj.strftime("%Y-%m-%d %H:%M:%S"),
"endTime": end_bj.strftime("%Y-%m-%d %H:%M:%S"),
}
if settings.JD_UNION_AUTH_KEY:
order_req["key"] = settings.JD_UNION_AUTH_KEY
result = call("jd.union.open.order.row.query", {"orderReq": order_req})
rows, has_more = _extract_rows(result)
logger.info(
"jd.union.open.order.row.query fetched rows=%s page=%s has_more=%s",
len(rows),
page_index,
has_more,
)
return {"rows": rows, "has_more": has_more, "raw": result}
+148
View File
@@ -0,0 +1,148 @@
"""穿山甲 GroMore「聚合数据报告 API」客户端 —— 按天拉取收益报表(只读、T+1)。
这是 **GroMore 数据拉取 API**,与发奖回调验签(integrations/pangle.py m-key)
两套完全不同的凭证与用途凭证在后台接入中心 GroMore-API 聚合数据报告 API领取:
媒体账号 user_id子账号 role_idSecurity Key(secure_key)
鉴权(文档 v2.x方法一):
1. 去掉请求参数里的 `sign` 字段与值为空的字段;
2. 其余参数按 key 字典序升序,拼成 `k1=v1&k2=v2&...&kn=vn`;
3. 末尾直接拼接 security_key(无分隔符),对整串做 MD5, 32 位小写十六进制 = sign
签名有 3 分钟过期(天级用 `timestamp` 秒级时间戳;小时级才用 `current_time` 字符串)
数据口径要点(来自官方文档):
- 只返回GroMore 聚合代码位 GroMore 内的数据(=我们 useMediation 的口径),
查不到穿山甲 SDK 自身的数据;
- **不提供分用户/设备维度**(官方 FAQ 明确拒绝),最细到 日期×应用×代码位×广告源;
- `revenue` = 预估收益(,所有 ADN 都有);`api_revenue` = 收益Api( ADN Reporting
回传按实时汇率折算账号币种,更接近结算),需后台为该 ADN 配置 Reporting 才有且不支持当天;
- 今天今天以前必须分开查;天级跨度 1 个月不早于 12 个月
"""
from __future__ import annotations
import hashlib
import logging
import time
from collections.abc import Mapping
from typing import Any
import httpx
from app.core.config import settings
logger = logging.getLogger("shagua.pangle_report")
HOST = "https://www.csjplatform.com"
# 天级收益报表(路径与文档代码示例一致;另有小时级 get_hour_report_data,本服务只用天级)。
DAILY_PATH = "/union_media/open/api/mediation/get_daily_income_report_data"
VERSION = "2.0"
SIGN_TYPE = "MD5"
PAGE_LIMIT = 5000 # 文档上限;一次尽量多取,减少翻页
DEFAULT_TIMEOUT = 30.0
_MAX_PAGES = 200 # 翻页安全阀(5000×200=100w 行,远超我们规模),防异常 has_next 死循环
class PangleReportError(Exception):
"""GroMore 报表接口调用失败(未配置 / 网络 / 业务码非 100)。"""
def build_sign(params: Mapping[str, Any], security_key: str) -> str:
"""按文档「方法一」生成 sign:去 sign/空值 → key 升序 → k=v& 拼接 → 末尾接 secure_key → MD5(32 位小写)。
与官方 Python/Java 示例逐字节一致( tests/test_pangle_report.py 的两个测试向量)
"""
items = [
(str(k), str(v))
for k, v in params.items()
if k != "sign" and v is not None and str(v) != ""
]
items.sort(key=lambda kv: kv[0])
raw = "&".join(f"{k}={v}" for k, v in items)
return hashlib.md5((raw + security_key).encode("utf-8")).hexdigest()
def fetch_daily_report(
*,
start_date: str,
end_date: str,
# ⚠️ 用 ad_unit_id(GroMore 广告位ID = 我们的 104xxx),不要用 code_id(底层各 ADN 代码位,对不上)
dimensions: str = "date,site_id,ad_unit_id",
metrics: str = "revenue,api_revenue,ecpm,imp_cnt",
site_ids: str | None = None,
code_ids: str | None = None,
os: str | None = None,
network: str | None = None,
time_zone: int = 8,
limit: int = PAGE_LIMIT,
timeout: float = DEFAULT_TIMEOUT,
) -> list[dict[str, Any]]:
"""拉一个日期区间(北京时间,time_zone=8)的天级收益报表,自动翻页,返回 report_list 全量行。
每行是字符串字典(接口原值),形如
{"start_date": "2026-06-27", "site_id": "5830519", "ad_unit_id": "104142227",
"revenue": "1.23", "api_revenue": "1.05", "ecpm": "0.80", "imp_cnt": "1537", ...}
业务码非 100 直接抛 PangleReportError(由调用方/脚本兜底,不静默吞)
"""
if not settings.pangle_report_configured:
raise PangleReportError(
"PANGLE_REPORT_USER_ID / ROLE_ID / SECURITY_KEY 未配置,无法拉取 GroMore 报表"
)
base_params: dict[str, Any] = {
"user_id": settings.PANGLE_REPORT_USER_ID,
"role_id": settings.PANGLE_REPORT_ROLE_ID,
"version": VERSION,
"sign_type": SIGN_TYPE,
"start_date": start_date,
"end_date": end_date,
"dimensions": dimensions,
"metrics": metrics,
"time_zone": time_zone,
"limit": limit,
}
# 可选过滤(空则不传,避免进签名串)
for key, val in (
("site_ids", site_ids), ("code_ids", code_ids),
("os", os), ("network", network),
):
if val:
base_params[key] = val
rows: list[dict[str, Any]] = []
offset = 0
try:
with httpx.Client(timeout=timeout) as client:
for _ in range(_MAX_PAGES):
params = dict(base_params)
params["offset"] = offset
# timestamp 每页临请求时取最新(3 分钟过期),并参与签名
params["timestamp"] = int(time.time())
params["sign"] = build_sign(params, settings.PANGLE_REPORT_SECURITY_KEY)
resp = client.get(HOST + DAILY_PATH, params=params)
resp.raise_for_status()
body = resp.json()
code = str(body.get("code"))
if code != "100":
raise PangleReportError(
f"GroMore 报表业务失败 code={code} message={body.get('message')!r} "
f"(101=验签失败/102=userid无效/107=无权限/118=无收益查看权限,详见文档状态码)"
)
data = body.get("data") or {}
page = data.get("report_list") or []
rows.extend(page)
# has_next=1 还有下一页;无该字段则按本页是否取满判断
has_next = str(data.get("has_next", "")) == "1"
if not page or not has_next:
break
offset += len(page)
except httpx.HTTPError as e:
raise PangleReportError(f"GroMore 报表请求异常: {e}") from e
logger.info(
"GroMore 天级报表拉取完成 %s~%s dims=%s 行数=%d", start_date, end_date, dimensions, len(rows)
)
return rows
+38 -10
View File
@@ -2,18 +2,23 @@
通过 `uvicorn app.main:app --reload` 启动
"""
from __future__ import annotations
import logging
from collections.abc import AsyncIterator
from contextlib import asynccontextmanager
from pathlib import Path
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import FileResponse
from fastapi.staticfiles import StaticFiles
from app.api.internal.app_version import router as internal_app_version_router
from app.api.internal.launch_confirm import router as internal_launch_confirm_router
from app.api.internal.price import router as internal_price_router
from app.api.internal.store import router as internal_store_router
from app.api.v1.ad import router as ad_router
from app.api.v1.analytics import router as analytics_router
from app.api.v1.auth import router as auth_router
@@ -21,12 +26,8 @@ from app.api.v1.compare import router as compare_router
from app.api.v1.compare_milestone import router as compare_milestone_router
from app.api.v1.compare_record import router as compare_record_router
from app.api.v1.coupon import router as coupon_router
from app.api.v1.device import router as device_router
from app.api.v1.cps_redirect import router as cps_redirect_router
from app.api.internal.app_version import router as internal_app_version_router
from app.api.internal.launch_confirm import router as internal_launch_confirm_router
from app.api.internal.price import router as internal_price_router
from app.api.internal.store import router as internal_store_router
from app.api.v1.device import router as device_router
from app.api.v1.feedback import router as feedback_router
from app.api.v1.invite import router as invite_router
from app.api.v1.meituan import router as meituan_router
@@ -40,15 +41,16 @@ from app.api.v1.user import router as user_router
from app.api.v1.wallet import router as wallet_router
from app.api.v1.wxpay import router as wxpay_router
from app.core.config import settings
from app.core.heartbeat_monitor_worker import (
start_heartbeat_monitor,
stop_heartbeat_monitor,
)
from app.core.daily_exchange_worker import (
start_daily_exchange_worker,
stop_daily_exchange_worker,
)
from app.core.heartbeat_monitor_worker import (
start_heartbeat_monitor,
stop_heartbeat_monitor,
)
from app.core.logging import setup_logging
from app.core.pricebot_client import aclose_pricebot_client, get_pricebot_client
from app.core.withdraw_reconcile_worker import (
start_withdraw_reconcile_worker,
stop_withdraw_reconcile_worker,
@@ -68,6 +70,7 @@ async def lifespan(_: FastAPI) -> AsyncIterator[None]:
settings.APP_DEBUG,
settings.DATABASE_URL.split("://", 1)[0],
)
get_pricebot_client() # 预热透传 client:把建 SSL 上下文的一次性成本付在启动,首个领券请求即热
reconcile_task = start_withdraw_reconcile_worker()
heartbeat_task = start_heartbeat_monitor()
daily_exchange_task = start_daily_exchange_worker()
@@ -77,6 +80,7 @@ async def lifespan(_: FastAPI) -> AsyncIterator[None]:
await stop_heartbeat_monitor(heartbeat_task)
await stop_withdraw_reconcile_worker(reconcile_task)
await stop_daily_exchange_worker(daily_exchange_task)
await aclose_pricebot_client()
logger.info("shutting down")
@@ -134,6 +138,30 @@ app.include_router(cps_redirect_router)
# 用户上传文件(头像)静态服务。生产可改由 nginx 直接 serve MEDIA_ROOT。
_media_root = Path(settings.MEDIA_ROOT)
_media_root.mkdir(parents=True, exist_ok=True)
# 官网下载的 APK 直链(落地页 dl.html「官网下载」按钮指向 /media/shaguabijia.apk)。
# 必须在 StaticFiles 挂载【之前】注册,否则被静态挂载吃掉。
# StaticFiles 给 .apk 的 Content-Type 不对、且无 attachment 头 → 部分国产浏览器不触发下载、转甩应用市场;
# 这里显式回 application/vnd.android.package-archive + Content-Disposition:attachment 强制浏览器下载。
# 文件由 scripts/publish_apk.sh 编 release 包后放到 data/media/shaguabijia.apk(*.apk 不入 git,需部署时放)。
_APK_PATH = _media_root / "shaguabijia.apk"
@app.get(f"{settings.MEDIA_URL_PREFIX}/shaguabijia.apk", tags=["meta"], include_in_schema=False)
def download_apk() -> FileResponse:
if not _APK_PATH.is_file():
from fastapi import HTTPException
raise HTTPException(status_code=404, detail="安装包未就绪")
return FileResponse(
_APK_PATH,
media_type="application/vnd.android.package-archive",
filename="shaguabijia.apk",
headers={"Content-Disposition": 'attachment; filename="shaguabijia.apk"'},
)
app.mount(
settings.MEDIA_URL_PREFIX,
StaticFiles(directory=str(_media_root)),
+2
View File
@@ -1,6 +1,7 @@
"""所有 ORM model 必须在这里 import 一次,Alembic / metadata 才能扫到。"""
from app.models.ad_ecpm import AdEcpmRecord # noqa: F401
from app.models.ad_feed_reward import AdFeedRewardRecord # noqa: F401
from app.models.ad_pangle_revenue import AdPangleDailyRevenue # noqa: F401
from app.models.ad_reward import AdRewardRecord # noqa: F401
from app.models.ad_watch_log import AdWatchLog # noqa: F401
from app.models.admin import AdminAuditLog, AdminUser # noqa: F401
@@ -18,6 +19,7 @@ from app.models.coupon_state import ( # noqa: F401
CouponClaimRecord,
CouponDailyCompletion,
CouponPromptEngagement,
CouponSession,
)
from app.models.feedback import Feedback # noqa: F401
from app.models.invite import InviteRelation # noqa: F401
+73
View File
@@ -0,0 +1,73 @@
"""穿山甲 GroMore 天级收益报表(后台结算口径,定时拉取入库)。
每行 = GroMore 数据 API 返回的一条日期 × 应用 × 代码位聚合收益(`integrations/pangle_report`
+ `scripts/sync_pangle_revenue` 落库)**权威/预估收益的来源**, `ad_ecpm_record`(客户端自报
eCPM 折算的预估)互为对照:
- `revenue_yuan` 接口 `revenue`(预估收益,;排序价×展示/1000,所有 ADN 都有);
- `api_revenue_yuan` 接口 `api_revenue`(收益Api,; ADN Reporting 回传更接近结算;
未配置该 ADN Reporting 或查当天时为空)
穿山甲不提供分用户/设备维度,故本表最细只到 日期×应用×代码位,**无法挂到逐条广告事件**;
广告收益报表里只用于汇总/趋势级的穿山甲后台收益,不改逐条行的客户端预估
"""
from __future__ import annotations
from datetime import datetime
from sqlalchemy import (
DateTime,
Float,
Integer,
String,
UniqueConstraint,
func,
)
from sqlalchemy.orm import Mapped, mapped_column
from app.db.base import Base
class AdPangleDailyRevenue(Base):
__tablename__ = "ad_pangle_daily_revenue"
__table_args__ = (
# 一行 = (日期, 应用, 代码位, 广告源);adn 用 "" 表示「未分广告源、该代码位汇总」,
# 避免 NULL 在唯一约束里被视为各不相同导致 upsert 重复(SQLite/PG 行为一致)。
UniqueConstraint(
"report_date", "app_env", "our_code_id", "adn",
name="uq_ad_pangle_daily",
),
)
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
# 北京时间日期串 'YYYY-MM-DD'(拉取时 time_zone=8),与 ad_ecpm_record.report_date 同口径,可直接 join。
report_date: Mapped[str] = mapped_column(String(10), index=True, nullable=False)
# 我们的应用环境:prod(傻瓜比价正式)/ test(测试);由 site_id 经 settings 映射而来。
app_env: Mapped[str] = mapped_column(String(16), nullable=False)
# 原始 GroMore AppId(site_id),留痕便于排查映射。
site_id: Mapped[str | None] = mapped_column(String(32), nullable=True)
# 广告位 ID(= 接口 ad_unit_id = 我们客户端配的 104xxx = ad_ecpm_record.our_code_id),join key。
# ⚠️ 不是接口的 code_id —— 那是底层各 ADN 的代码位(如 983674557),对不上我们的口径。
our_code_id: Mapped[str] = mapped_column(String(64), index=True, nullable=False)
# 广告源(接口 network 数字→名,如 pangle/gdt);"" = 未分广告源的代码位汇总行(当前默认口径)。
adn: Mapped[str] = mapped_column(String(16), nullable=False, default="")
# 预估收益(元)← 接口 revenue。
revenue_yuan: Mapped[float] = mapped_column(Float, nullable=False, default=0.0)
# 收益Api(元)← 接口 api_revenue;未配 Reporting / 当天 等情况接口不返回 → NULL。
api_revenue_yuan: Mapped[float | None] = mapped_column(Float, nullable=True)
# 预估 eCPM 原值(接口 ecpm,单位元/千次,**与客户端 getEcpm 的「分」不同**),参考用原样存。
ecpm: Mapped[str | None] = mapped_column(String(32), nullable=True)
# 展示次数 ← 接口 imp_cnt。
impressions: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
# 货币类型(接口 currency,正常为 cny)。
currency: Mapped[str] = mapped_column(String(8), nullable=False, default="cny")
# 最近一次同步写入时间(同一行可被多次回补覆盖;T+1 数据穿山甲会订正)。
synced_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), onupdate=func.now(), nullable=False
)
def __repr__(self) -> str: # pragma: no cover
return (
f"<AdPangleDailyRevenue {self.report_date} {self.app_env} "
f"code={self.our_code_id} revenue={self.revenue_yuan}>"
)
+76
View File
@@ -181,3 +181,79 @@ class CouponPromptEngagement(Base):
f"<CouponPromptEngagement device={self.device_id} "
f"date={self.engage_date} type={self.engage_type}>"
)
class CouponSession(Base):
"""一次领券任务的全程流水(admin「领券数据」看板数据源,2026-06-30)。
coupon_claim_record(按券一天一条去重)coupon_daily_completion(按设备一天一条)都不同:
本表**一次领券一条**(trace_id 唯一),记从发起(started)到收尾(completed/failed/abandoned)
全程耗时 + 各平台耗时 + 机型/ROM客户端 POST /api/v1/coupon/session 两段上报:发起建行
收尾按 trace_id 更新同一行发起即落库 admin 可算发起数与中途流失(started 无终态=未完成)
口径:elapsed_ms 由客户端全程计时(点发起收尾)权威;started_at/finished_at 为时刻留痕
started_date = started_at Asia/Shanghai 自然日, admin 按天聚合 / 日期筛选(索引)
"""
__tablename__ = "coupon_session"
__table_args__ = (
# 一次领券一行:trace_id 幂等 upsert(发起建、收尾更新同一行)。
UniqueConstraint("trace_id", name="uq_coupon_session_trace"),
# admin 主聚合/筛选:按上海自然日 + 环境(报表默认只看 prod,避免测试数据串台)。
Index("ix_coupon_session_date_env", "started_date", "app_env"),
)
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
# 一次领券唯一 id(客户端 UUID,全程贯穿),upsert 键。
trace_id: Mapped[str] = mapped_column(String(64), nullable=False)
device_id: Mapped[str] = mapped_column(String(64), nullable=False)
# 登录态才带(admin join 用户表出手机号/昵称);匿名领券为空。
user_id: Mapped[int | None] = mapped_column(Integer, index=True, nullable=True)
# started / completed / failed / abandoned。started 无终态 = 中途流失(未完成)。
status: Mapped[str] = mapped_column(String(16), nullable=False)
# prod / dev(客户端 BuildConfig.DEBUG)。admin 报表默认只看 prod(对齐广告报表防串台口径)。
app_env: Mapped[str | None] = mapped_column(String(16), index=True, nullable=True)
# 发起勾选平台 ["meituan-waimai", ...](空=全领)。
platforms: Mapped[list | None] = mapped_column(_JSON, nullable=True)
# 发起来源外卖 App 包名;null=App 内(傻瓜比价首页)发起,非空=从美团/淘宝/京东弹券发起。
# admin「发起平台」列据此区分(空→傻瓜比价,包名→对应平台)。
origin_package: Mapped[str | None] = mapped_column(String(64), nullable=True)
# 机型(Build.MANUFACTURER + MODEL)与 ROM(OemDetector,如 "ColorOS 14")。明细「机型/ROM」列 + 维度筛选。
device_model: Mapped[str | None] = mapped_column(String(128), nullable=True)
rom: Mapped[str | None] = mapped_column(String(64), nullable=True)
# 发起时刻(客户端墙钟):明细「时间」列、趋势 X 轴。
started_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
# 发起的 Asia/Shanghai 自然日:按天聚合 / 日期范围筛选(索引)。
started_date: Mapped[date] = mapped_column(Date, nullable=False)
# 收尾时刻(服务端 now);未收尾(流失)为空。
finished_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True
)
# 全程耗时(ms,客户端点发起→收尾):平均 / 分位都基于它(只统计 completed)。
elapsed_ms: Mapped[int | None] = mapped_column(Integer, nullable=True)
# 各平台领券耗时 {"meituan-waimai":3200,...}(ms)。明细美团/淘宝/京东耗时列。
platform_elapsed: Mapped[dict | None] = mapped_column(_JSON, nullable=True)
# 领到总张数(收尾帧带)。
claimed_count: Mapped[int | None] = mapped_column(Integer, nullable=True)
# pricebot done 帧回传的公网调试链接(price.shaguabijia.com/traces/{dir});含落盘时分秒、拼不出,只能存
# (同 ComparisonRecord.trace_url)。admin「领券数据」明细据此渲染可点 trace 链接;未到 done(failed/abandoned)为空。
trace_url: Mapped[str | None] = mapped_column(String(512), nullable=True)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), nullable=False
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), onupdate=func.now(),
nullable=False,
)
def __repr__(self) -> str: # pragma: no cover
return (
f"<CouponSession trace={self.trace_id} status={self.status} "
f"elapsed_ms={self.elapsed_ms}>"
)
+22 -4
View File
@@ -1,11 +1,13 @@
"""CPS 对账订单(cps_order)。
美团联盟 query_order 按时间窗拉回 sid 归群的订单明细字段对齐 query_order
从联盟 API 按时间窗拉回平台落库的 CPS 订单明细字段最初对齐美团 query_order,
后续兼容京东订单报表:
实测返回:
- payPrice / profit 字符串 入库统一转(与全站口径一致)
- payTime / updateTime 是秒级时间戳 入库转 tz-aware datetime
- status: 2付款 3完成 4取消 5风控 6结算(取消/风控不计佣金)
order_id 全局唯一,reconcile 按它 upsert(订单状态会变,重复拉则更新)
order_id 全局唯一,reconcile 按它 upsert(订单状态会变,重复拉则更新)京东订单用
`jd:<row_id>` 前缀避免与美团订单号碰撞
"""
from __future__ import annotations
@@ -22,8 +24,13 @@ class CpsOrder(Base):
__tablename__ = "cps_order"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
# 美团订单号(加密串),全局唯一,upsert 幂等键
# 平台:meituan / jd。历史数据迁移默认 meituan
platform: Mapped[str] = mapped_column(String(20), default="meituan", index=True, nullable=False)
# 平台订单号/行号包装后的全局唯一键,upsert 幂等。
order_id: Mapped[str] = mapped_column(String(64), unique=True, index=True, nullable=False)
# 平台原始订单号/行号。京东一笔订单多 SKU 时可按行号区分。
external_order_id: Mapped[str | None] = mapped_column(String(128), index=True, nullable=True)
external_row_id: Mapped[str | None] = mapped_column(String(128), index=True, nullable=True)
# 渠道追踪位 = 群 sid(历史无 sid 订单为空)。按它归群聚合。
sid: Mapped[str | None] = mapped_column(String(64), index=True, nullable=True)
act_id: Mapped[str | None] = mapped_column(String(64), index=True, nullable=True)
@@ -35,11 +42,21 @@ class CpsOrder(Base):
commission_rate: Mapped[str | None] = mapped_column(String(16), nullable=True) # "300"=3% "10"=0.1%
refund_price_cents: Mapped[int | None] = mapped_column(Integer, nullable=True)
refund_profit_cents: Mapped[int | None] = mapped_column(Integer, nullable=True)
# 通用佣金拆分。美团只有预估 profit;京东有预估/实际佣金。
estimated_commission_cents: Mapped[int | None] = mapped_column(Integer, nullable=True)
actual_commission_cents: Mapped[int | None] = mapped_column(Integer, nullable=True)
# 美团订单状态: 2付款 3完成 4取消 5风控 6结算
mt_status: Mapped[str | None] = mapped_column(String(8), index=True, nullable=True)
# 京东订单有效码(validCode),用于判断是否有效/已完成。
jd_valid_code: Mapped[str | None] = mapped_column(String(16), index=True, nullable=True)
invalid_reason: Mapped[str | None] = mapped_column(String(128), nullable=True)
product_name: Mapped[str | None] = mapped_column(String(512), nullable=True)
settle_month: Mapped[str | None] = mapped_column(String(16), nullable=True)
site_id: Mapped[str | None] = mapped_column(String(128), nullable=True)
position_id: Mapped[str | None] = mapped_column(String(128), nullable=True)
pid: Mapped[str | None] = mapped_column(String(128), nullable=True)
sub_union_id: Mapped[str | None] = mapped_column(String(128), nullable=True)
pay_time: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), index=True, nullable=True
@@ -59,5 +76,6 @@ class CpsOrder(Base):
def __repr__(self) -> str: # pragma: no cover
return (
f"<CpsOrder id={self.id} order_id={self.order_id!r} "
f"sid={self.sid!r} status={self.mt_status} profit_cents={self.commission_cents}>"
f"platform={self.platform!r} sid={self.sid!r} "
f"status={self.mt_status or self.jd_valid_code} profit_cents={self.commission_cents}>"
)
+5
View File
@@ -25,6 +25,11 @@ class Feedback(Base):
contact: Mapped[str] = mapped_column(String(128), nullable=False)
# 截图 URL 列表(相对路径,如 ["/media/feedback/u1_ab12.jpg"]);无图为 None
images: Mapped[list[str] | None] = mapped_column(JSON, nullable=True)
# 提交时的端环境快照(admin 排查用;客户端改版带上后的新反馈才有,历史数据为 NULL)
app_version: Mapped[str | None] = mapped_column(String(32), nullable=True) # 我们 app versionName
device_model: Mapped[str | None] = mapped_column(String(64), nullable=True) # Build.MODEL
rom_name: Mapped[str | None] = mapped_column(String(32), nullable=True) # OemDetector os:ColorOS/MIUI/...
android_version: Mapped[str | None] = mapped_column(String(16), nullable=True) # Build.VERSION.RELEASE
# pending(审核中) / adopted(已采纳) / rejected(未采纳)
status: Mapped[str] = mapped_column(String(16), nullable=False, default="pending", index=True)
reject_reason: Mapped[str | None] = mapped_column(String(256), nullable=True)
+109
View File
@@ -0,0 +1,109 @@
"""穿山甲 GroMore 天级收益 读写(`ad_pangle_daily_revenue` 表)。
`scripts/sync_pangle_revenue` 拉数后调 `upsert_daily_rows` 落库(同一(日期×应用×代码位×广告源)
幂等覆盖,T+1 订正可重跑);admin 广告收益报表调 `aggregate_by_date` 穿山甲后台收益
汇总/趋势级展示穿山甲无用户维度,故这里不涉及 user_id
"""
from __future__ import annotations
from typing import Any, TypedDict
from sqlalchemy import func, select
from sqlalchemy.orm import Session
from app.models.ad_pangle_revenue import AdPangleDailyRevenue
# upsert 时可被覆盖更新的列(唯一键之外的业务列)。
_UPDATABLE = ("site_id", "revenue_yuan", "api_revenue_yuan", "ecpm", "impressions", "currency")
class PangleDateAgg(TypedDict):
date: str
revenue_yuan: float
api_revenue_yuan: float | None
impressions: int
def upsert_daily_rows(db: Session, rows: list[dict[str, Any]]) -> dict[str, int]:
"""按唯一键 (report_date, app_env, our_code_id, adn) 逐行 upsert。
每个 row 须含:report_date, app_env, our_code_id;可选 adn(默认"")site_id
revenue_yuanapi_revenue_yuanecpmimpressionscurrency返回 {inserted, updated}
规模很小(代码位数×天数),逐行 select-then-write 足够, SQLite/PG 通用
"""
inserted = updated = 0
for row in rows:
adn = row.get("adn") or ""
existing = db.execute(
select(AdPangleDailyRevenue).where(
AdPangleDailyRevenue.report_date == row["report_date"],
AdPangleDailyRevenue.app_env == row["app_env"],
AdPangleDailyRevenue.our_code_id == row["our_code_id"],
AdPangleDailyRevenue.adn == adn,
)
).scalar_one_or_none()
if existing is None:
db.add(AdPangleDailyRevenue(
report_date=row["report_date"],
app_env=row["app_env"],
site_id=row.get("site_id"),
our_code_id=row["our_code_id"],
adn=adn,
revenue_yuan=float(row.get("revenue_yuan") or 0.0),
api_revenue_yuan=row.get("api_revenue_yuan"),
ecpm=row.get("ecpm"),
impressions=int(row.get("impressions") or 0),
currency=row.get("currency") or "cny",
))
inserted += 1
else:
for col in _UPDATABLE:
if col in row:
setattr(existing, col, row[col])
updated += 1
db.commit()
return {"inserted": inserted, "updated": updated}
def aggregate_by_date(
db: Session,
*,
date_from: str,
date_to: str,
app_env: str | None = None,
our_code_id: str | None = None,
) -> list[PangleDateAgg]:
"""按日期汇总穿山甲收益(闭区间,北京时间),供报表趋势 + 合计。
revenue_yuan = Σrevenue;api_revenue_yuan = Σapi_revenue(SQL SUM 忽略 NULL,
全为空则返回 None,前端显示-)可选按应用 / 代码位过滤返回按日期升序
"""
stmt = (
select(
AdPangleDailyRevenue.report_date,
func.sum(AdPangleDailyRevenue.revenue_yuan),
func.sum(AdPangleDailyRevenue.api_revenue_yuan),
func.sum(AdPangleDailyRevenue.impressions),
)
.where(
AdPangleDailyRevenue.report_date >= date_from,
AdPangleDailyRevenue.report_date <= date_to,
)
.group_by(AdPangleDailyRevenue.report_date)
.order_by(AdPangleDailyRevenue.report_date)
)
if app_env is not None:
stmt = stmt.where(AdPangleDailyRevenue.app_env == app_env)
if our_code_id is not None:
stmt = stmt.where(AdPangleDailyRevenue.our_code_id == our_code_id)
out: list[PangleDateAgg] = []
for report_date, rev, api_rev, imp in db.execute(stmt).all():
out.append(PangleDateAgg(
date=report_date,
revenue_yuan=round(float(rev or 0.0), 6),
api_revenue_yuan=(round(float(api_rev), 6) if api_rev is not None else None),
impressions=int(imp or 0),
))
return out
+90 -1
View File
@@ -6,7 +6,7 @@
from __future__ import annotations
import logging
from datetime import date, datetime
from datetime import date, datetime, timezone
from zoneinfo import ZoneInfo
from sqlalchemy import delete, func, select
@@ -17,6 +17,7 @@ from app.models.coupon_state import (
CouponClaimRecord,
CouponDailyCompletion,
CouponPromptEngagement,
CouponSession,
)
logger = logging.getLogger("shagua.coupon_state")
@@ -232,3 +233,91 @@ def sum_claimed_count(db: Session, user_id: int) -> int:
)
).scalar_one()
return int(total or 0)
# ===== 领券任务流水(coupon_session,admin「领券数据」看板数据源)=====
def upsert_coupon_session(
db: Session,
*,
trace_id: str,
device_id: str,
status: str,
started_at_ms: int,
user_id: int | None = None,
platforms: list[str] | None = None,
origin_package: str | None = None,
device_model: str | None = None,
rom: str | None = None,
app_env: str | None = None,
elapsed_ms: int | None = None,
platform_elapsed: dict | None = None,
claimed_count: int | None = None,
trace_url: str | None = None,
) -> None:
"""一条领券流水按 trace_id 幂等 upsert(发起 started 建行、收尾终态更新同一行)。
- 乱序/重复兜底:终态(completed/failed/abandoned)先到也建行;started 重复到不覆盖已有终态
(状态只前进,不降级)
- started_at 由客户端墙钟毫秒转;started_date 取其 Asia/Shanghai 自然日(admin 按天聚合/筛选)
- 终态帧补 finished_at=服务端 now;各字段非空才写(避免 started 帧的 None 抹掉收尾值,反之亦然)
并发 IntegrityError 回滚忽略(本就幂等)
"""
started_at = datetime.fromtimestamp(started_at_ms / 1000, tz=timezone.utc)
started_date = started_at.astimezone(_CN_TZ).date()
is_terminal = status in ("completed", "failed", "abandoned")
row = db.execute(
select(CouponSession).where(CouponSession.trace_id == trace_id)
).scalar_one_or_none()
if row is None:
db.add(CouponSession(
trace_id=trace_id,
device_id=device_id,
user_id=user_id,
status=status,
app_env=app_env,
platforms=platforms,
origin_package=origin_package,
device_model=device_model,
rom=rom,
started_at=started_at,
started_date=started_date,
finished_at=datetime.now(timezone.utc) if is_terminal else None,
elapsed_ms=elapsed_ms,
platform_elapsed=platform_elapsed,
claimed_count=claimed_count,
trace_url=trace_url,
))
else:
# 状态只前进:started 帧重复到(如 START_STICKY 重启)不把已有终态降级回 started。
if not (status == "started" and row.status in ("completed", "failed", "abandoned")):
row.status = status
if is_terminal:
row.finished_at = datetime.now(timezone.utc)
if user_id is not None:
row.user_id = user_id
if platforms is not None:
row.platforms = platforms
if origin_package is not None:
row.origin_package = origin_package
if device_model is not None:
row.device_model = device_model
if rom is not None:
row.rom = rom
if app_env is not None:
row.app_env = app_env
if elapsed_ms is not None:
row.elapsed_ms = elapsed_ms
if platform_elapsed is not None:
row.platform_elapsed = platform_elapsed
if claimed_count is not None:
row.claimed_count = claimed_count
if trace_url is not None:
row.trace_url = trace_url
try:
db.commit()
except IntegrityError:
# 并发下另一请求刚插了同 trace_id → 唯一约束撞,回滚忽略(本就幂等)。
db.rollback()
+8
View File
@@ -17,12 +17,20 @@ def create_feedback(
content: str,
contact: str,
images: list[str] | None,
app_version: str | None = None,
device_model: str | None = None,
rom_name: str | None = None,
android_version: str | None = None,
) -> Feedback:
fb = Feedback(
user_id=user_id,
content=content,
contact=contact,
images=images or None,
app_version=app_version or None,
device_model=device_model or None,
rom_name=rom_name or None,
android_version=android_version or None,
status="pending",
created_at=datetime.now(CN_TZ).replace(tzinfo=None),
)
+9 -1
View File
@@ -216,13 +216,21 @@ def try_reward_on_compare(db: Session, invitee_user_id: int) -> CompareRewardRes
def get_stats(db: Session, inviter_id: int) -> tuple[int, int]:
"""返回 (已成功邀请人数, 累计从邀请获得的金币)。
"已邀请好友数"口径 = 完成一次比价(已触发邀请奖励金)的被邀请人数, compare_reward_granted=True
不再数"仅绑定未比价"的关系否则会出现"已邀请 1、可提现余额 0"(好友下载登录但没比价),
与产品口径"已邀请好友数 × 2元 = 累计提现 + 可提现余额"对不上过滤后该恒等式天然成立
(每个计入的好友都恰好发过 1 2 ,钱要么在余额要么已提现)
金币口径(inviter_coin 之和) v3 起恒 0(邀请人收益改走邀请奖励金, get_reward_stats /
try_reward_on_compare);保留返回位兼容旧响应字段 coins_earned
"""
count = db.execute(
select(func.count())
.select_from(InviteRelation)
.where(InviteRelation.inviter_user_id == inviter_id)
.where(
InviteRelation.inviter_user_id == inviter_id,
InviteRelation.compare_reward_granted.is_(True),
)
).scalar_one()
coins = db.execute(
select(func.coalesce(func.sum(InviteRelation.inviter_coin), 0))
+26
View File
@@ -2,8 +2,10 @@
from __future__ import annotations
import logging
from datetime import datetime
from typing import Optional
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.models.launch_confirm_sample import LaunchConfirmSample
@@ -33,3 +35,27 @@ def insert_sample(db: Session, payload: LaunchConfirmSampleIn) -> int:
db.commit()
db.refresh(row)
return row.id
def list_samples(
db: Session,
*,
exec_success: Optional[bool] = None,
host_package: Optional[str] = None,
since: Optional[datetime] = None,
limit: int = 1000,
) -> list[LaunchConfirmSample]:
"""按条件查样本(pricebot 的 distill_launch_confirm.py 沉淀工具读)。created_at 升序。
过滤项都可空:exec_success(只看放行成功的)/ host_package(只看某宿主包)/
since(>= created_at)limit 兜底防一次拉爆全表
"""
stmt = select(LaunchConfirmSample)
if exec_success is not None:
stmt = stmt.where(LaunchConfirmSample.exec_success == exec_success)
if host_package:
stmt = stmt.where(LaunchConfirmSample.host_package == host_package)
if since is not None:
stmt = stmt.where(LaunchConfirmSample.created_at >= since)
stmt = stmt.order_by(LaunchConfirmSample.created_at.asc()).limit(limit)
return list(db.execute(stmt).scalars().all())
+7 -1
View File
@@ -610,6 +610,7 @@ def create_withdraw(
source: str = "coin_cash",
user_name: str | None = None,
out_bill_no: str | None = None,
allow_sub_min: bool = False,
) -> WithdrawOrder:
"""发起提现:原子扣款 + 建单 reviewing(待人工审核),**不打款**。
@@ -619,8 +620,13 @@ def create_withdraw(
重复发起多笔提现(审核拒绝再退回)
#2 out_bill_no 客户端幂等键:同号重试返回该单现状(reviewing 等审核),不重复扣款建单。
实名 user_name 在此存下(WithdrawOrder.user_name),供异步审核打款时传给微信(达额需实名)
allow_sub_min:放行低于"提现最低额"的小额(用于 0.01 元调试提现)仅由 endpoint
`skip_review and not is_prod`(debug + 非生产双闸)时置 True;仍受 schema gt=0 max 上限约束
生产恒为 False 最低额校验照常,绝不可能提 0.01
"""
if amount_cents < rewards.get_withdraw_min_cents(db) or amount_cents > rewards.get_withdraw_max_cents(db):
min_c = 0 if allow_sub_min else rewards.get_withdraw_min_cents(db)
if amount_cents < min_c or amount_cents > rewards.get_withdraw_max_cents(db):
raise InvalidWithdrawAmountError
# 提现即要求已绑微信:否则审核通过也打不了款,提前拦更友好
+2
View File
@@ -123,6 +123,7 @@ class ComparisonRecordIn(BaseModel):
# pricebot done.params.trace_url 原样上报,落库供记录页「复制调试链接」(dir 名含落盘
# 时分秒前端拼不出,必须由后端透传)。
trace_url: str | None = Field(None, description="本次比价公网调试链接")
total_ms: int | None = Field(None, description="整场比价墙钟耗时(ms)")
# ===== debug 维度(客户端采集上报;旧客户端不带 → None。仅 admin 比价记录页用)=====
# 必须显式声明,否则 model_dump() 落 raw_payload 时被 pydantic 静默丢弃(同上面 coupon_saved 的坑)。
@@ -172,6 +173,7 @@ class ComparisonRecordOut(BaseModel):
items: list = []
comparison_results: list = []
skipped_dish_names: list = []
total_ms: int | None = None
# 「已下单」(店级):该店名在该用户真实下单(source='compare')里出现过即 True。
# 由 list_records 动态算出挂在 ORM 实例上(非 DB 列),from_attributes 读出;缺省 False。
ordered: bool = False
+25
View File
@@ -49,3 +49,28 @@ class CouponStatsOut(BaseModel):
"""
coupon_count: int
class CouponSessionIn(BaseModel):
"""客户端领券流水上报体(admin「领券数据」看板数据源,POST /api/v1/coupon/session)。
一次领券两段上报, trace_id upsert coupon_session:
- 发起(status=started):带勾选平台 + 机型/ROM/app_env + started_at_ms(发起墙钟毫秒)
- 收尾(completed/failed/abandoned): elapsed_ms(全程耗时)+ platform_elapsed(各平台耗时)+ claimed_count
不鉴权(同领券循环 MVP, device_id/trace_id),user_id 登录态带上做留痕(可空)
"""
trace_id: str
device_id: str
status: str # started / completed / failed / abandoned
started_at_ms: int # 发起墙钟毫秒(客户端 System.currentTimeMillis)
user_id: int | None = None
platforms: list[str] | None = None
origin_package: str | None = None
device_model: str | None = None
rom: str | None = None
app_env: str | None = None
elapsed_ms: int | None = None
platform_elapsed: dict[str, int] | None = None
claimed_count: int | None = None
trace_url: str | None = None
+20 -1
View File
@@ -1,7 +1,9 @@
"""启动确认窗兜底样本的内部上报模型(pricebot → app-server)。"""
from __future__ import annotations
from pydantic import BaseModel
from datetime import datetime
from pydantic import BaseModel, ConfigDict
class LaunchConfirmSampleIn(BaseModel):
@@ -24,3 +26,20 @@ class LaunchConfirmSampleOut(BaseModel):
"""落库结果。"""
id: int
class LaunchConfirmSampleRow(BaseModel):
"""单条样本(列表读出;沉淀脚本 distill 用,payload 原样带出)。"""
model_config = ConfigDict(from_attributes=True)
id: int
created_at: datetime
trace_id: str | None = None
device_id: str | None = None
host_package: str | None = None
target_app: str | None = None
system_locale: str | None = None
exec_success: bool = False
dialog_title: str | None = None
payload: dict | None = None
Binary file not shown.

After

Width:  |  Height:  |  Size: 847 KiB

+13 -4
View File
@@ -198,7 +198,7 @@
选择「<span class="guide-highlight">在浏览器打开</span>
<span class="guide-final">在浏览器里按提示<span class="guide-target">去应用商店下载</span></span>
</h2>
<div class="guide-dismiss" id="wxGuideDismiss">我知道了 ✕</div>
<!-- <div class="guide-dismiss" id="wxGuideDismiss">我知道了 ✕</div> -->
</div>
<div class="toast" id="toast" role="status" aria-live="polite"></div>
@@ -263,7 +263,7 @@
var wxGuide = document.getElementById("wxGuide");
function showWxGuide() { wxGuide.classList.add("show"); }
function hideWxGuide() { wxGuide.classList.remove("show"); }
document.getElementById("wxGuideDismiss").addEventListener("click", hideWxGuide);
// document.getElementById("wxGuideDismiss").addEventListener("click", hideWxGuide); // 「我知道了 ✕」已注释隐藏
if (isWeChat) showWxGuide(); // 微信里一进页面就提示去浏览器(微信内下载必被拦)
function showToast(text) {
@@ -273,15 +273,24 @@
showToast.timer = setTimeout(function () { toast.classList.remove("show"); }, 1400);
}
// ===== 下载按钮:微信内引导去浏览器;iOS 提示;安卓写邀请码 + 跳应用商店 =====
// ===== 应用商店下载:微信内引导去浏览器;iOS 提示;安卓写邀请码 + 跳应用商店 =====
function handleDownload() {
if (isWeChat) { showWxGuide(); return; }
if (isIOS) { alert("iOS 版即将上线,请前往 App Store 搜索「傻瓜比价」"); return; }
copyInviteCode(); // 先把邀请码写进剪贴板(供 App 首启归因),链路丢了还有 landing-track 指纹兜底
openStore();
}
// ===== 官网下载:微信/iOS 同上引导;安卓直接跳 APK 直链 → 弹系统下载弹窗 =====
// APK_URL 跟随页面 host:本地走 LAN、生产走 app-api.shaguabijia.com(见邀请功能文档约定)。
var APK_URL = location.origin + "/media/shaguabijia.apk";
function handleWebsiteDownload() {
if (isWeChat) { showWxGuide(); return; }
if (isIOS) { alert("iOS 版即将上线,请前往 App Store 搜索「傻瓜比价」"); return; }
copyInviteCode(); // 同样先写邀请码进剪贴板(供 App 首启归因)
window.location.href = APK_URL;
}
document.getElementById("dlbtn").addEventListener("click", handleDownload);
document.getElementById("dlbtn2").addEventListener("click", handleDownload);
document.getElementById("dlbtn2").addEventListener("click", handleWebsiteDownload);
</script>
</body>
</html>
Binary file not shown.

After

Width:  |  Height:  |  Size: 35 KiB

+67
View File
@@ -0,0 +1,67 @@
# 穿山甲 GroMore 收益拉取 定时任务 — 运维手册
> 对象:维护「每天拉穿山甲后台收益入库」这套定时任务的同事。
> 🔒 服务器登录信息见**私密交接清单**,不入库。
## 它是什么
admin「广告收益报表」里的「穿山甲后台收益(T+1)」读的是**本地表 `ad_pangle_daily_revenue` 的快照,不是实时查穿山甲**。穿山甲只通过 GroMore 数据 API 给数、且 **T+1**(次日约 10:00 出昨天的数),所以每天得拉一次入库,报表才会往前走。
- 每天 10:30 跑一轮 `scripts/sync_pangle_revenue.py`,默认 `--days 3` 回补近 3 天。
- 维度 = 日期 × 应用(site_id)× 广告位(ad_unit_id);指标 = `revenue`(预估)+ `api_revenue`(结算口径)。
- **幂等 upsert**:同一(日期×应用×代码位)重跑只覆盖、不重复,故回补 / 重跑 / catch-up 都安全。
- 穿山甲无用户/设备维度 → 只能落「汇总/趋势级」,报表带 user_id 过滤时这块收益置空(显示「-」)。
## 文件
| 项 | 路径 |
|---|---|
| 脚本入口 | `scripts/sync_pangle_revenue.py` |
| 拉取 / 签名 | `app/integrations/pangle_report.py`(签名=参数字典序拼接+secure_key 后 MD5) |
| 入库表 | `ad_pangle_daily_revenue`(读写在 `app/repositories/ad_pangle_revenue.py`) |
| 凭证 | `.env``PANGLE_REPORT_USER_ID` / `PANGLE_REPORT_ROLE_ID` / `PANGLE_REPORT_SECURITY_KEY` |
| 应用映射 | `.env``PANGLE_REPORT_SITE_ID_PROD`(5830519)/ `PANGLE_REPORT_SITE_ID_TEST`(5832303) |
| systemd 单元 | `deploy/pangle-revenue.{service,timer}` |
## 上线前置(只做一次)
1. **填凭证**:后台「接入中心 → GroMore-API」领 user_id / role_id / Security Key(`secure_key`,**≠ 发奖 m-key**),填进线上 `.env``PANGLE_REPORT_*`。三项填齐脚本才工作,缺任一 → no-op 退出。
2. **子账号要授权**:线上若用子账号(role_id ≠ user_id),需主账号给它授「查看全部数据」,否则接口报 **118**(无权限)。本人自查自己(user_id=role_id)无此问题。
3. 凭证泄露:后台「重置 key 值」即可,改完同步线上 `.env` 重跑。
## 部署(Linux 服务器,需 root)
```bash
sudo cp deploy/pangle-revenue.{service,timer} /etc/systemd/system/
sudo systemctl daemon-reload && sudo systemctl enable --now pangle-revenue.timer
systemctl list-timers pangle-revenue.timer # 确认下次触发时间(应是次日 10:30)
```
## 怎么看健康 / 手动跑一次
```bash
sudo systemctl start pangle-revenue.service # 立即手动跑一轮(不等 10:30)
journalctl -u pangle-revenue -n 30 --no-pager # 看日志:拉取区间 / 入库行数 / 新增更新 / 预估收益合计
```
成功日志形如:`✅ 完成:接口 N 行 → 入库 M 行(跳过 x),新增 a / 更新 b;预估收益合计 ¥19.42`
> 看不到收益、提示 `PANGLE_REPORT_* 未配置`→ 回「上线前置」补 `.env`;报 118 → 子账号没授「查看全部数据」。
## 本机 Windows 开发(无 systemd)
直接手动跑:
```
.venv\Scripts\python -m scripts.sync_pangle_revenue # 拉昨天
.venv\Scripts\python -m scripts.sync_pangle_revenue --days 3 # 回补近 3 天
.venv\Scripts\python -m scripts.sync_pangle_revenue --date 2026-06-27 # 指定单天
.venv\Scripts\python -m scripts.sync_pangle_revenue --start 2026-06-01 --end 2026-06-27 # 区间回补
```
## 脚本参数
- 无参:拉**昨天**(北京时间)。timer 用的是 `--days 3`
- `--days N`:从昨天起往前回补 N 天(含昨天)。
- `--date YYYY-MM-DD`:指定单天。
- `--start / --end`:指定闭区间(跨度 ≤ 31 天,接口上限 1 个月,超了报 114)。
## 注意事项
- **触发时间**:`OnCalendar=*-*-* 10:30:00`。穿山甲 ~10:00 出数,故别早于 10:00 跑(会拉到空/不全)。
- **catch-up**:`Persistent=true` 补跑错过的那一轮;叠加 `--days 3`,漏一两天重新触发即自愈。
- **今天 / 今天以前要分开查**:脚本默认只拉昨天及更早,不混查今天(接口约束),无需关心。
- **join key 是 `ad_unit_id`(我们配的 104xxx)不是 `code_id`**:`code_id` 是底层各 ADN 代码位,对不上口径;`ad_unit_id='-1'` 是未归因桶。改维度时务必注意(详见脚本头注释)。
- **`api_revenue` 很稀疏**:测试应用 ADN 没配 Reporting → 全 0,仅 prod 个别位有;`revenue`(预估)才是稳的主力。
- **DB 无关**:sqlite / postgres 均可(upsert 逐行 select-then-write,不像美团 ETL 需要 PG)。
- **别和别的触发方式双跑**:本 systemd timer 与「手动 cron / 进程内任务」二选一,虽幂等不会重复入库,纯属多余。
- **改脚本 / 改部署**:走 git + PR,由有 root 的人部署。
+38
View File
@@ -0,0 +1,38 @@
# 每天拉穿山甲 GroMore T+1 天级收益入库 —— 单轮跑,由 pangle-revenue.timer 每天 10:30 触发。
# 落 ad_pangle_daily_revenue 表,供 admin 广告收益报表的「穿山甲后台收益(T+1)」区块。
#
# 仅用于 Linux 服务器;本机 Windows 开发无 systemd,直接手动跑脚本即可:
# .venv\Scripts\python -m scripts.sync_pangle_revenue # 拉昨天(北京时间)
# .venv\Scripts\python -m scripts.sync_pangle_revenue --days 3 # 回补近 3 天
#
# 部署(服务器):
# sudo cp deploy/pangle-revenue.{service,timer} /etc/systemd/system/
# sudo systemctl daemon-reload && sudo systemctl enable --now pangle-revenue.timer
# # 手动跑一次验证: sudo systemctl start pangle-revenue.service && journalctl -u pangle-revenue -n 30
#
# 前置:.env 配好 PANGLE_REPORT_USER_ID / PANGLE_REPORT_ROLE_ID / PANGLE_REPORT_SECURITY_KEY
# (后台「接入中心 → GroMore-API」领;子账号需主账号授「查看全部数据」否则接口 118)。
# 未配齐这三项 → 脚本自动 no-op 退出,免动 timer。
[Unit]
Description=Sync Pangle GroMore daily revenue (T+1, one-shot, driven by timer)
After=network-online.target
Wants=network-online.target
[Service]
Type=oneshot
User=root
WorkingDirectory=/opt/shaguabijia-app-server
Environment="PATH=/opt/shaguabijia-app-server/.venv/bin:/usr/bin:/bin"
EnvironmentFile=/opt/shaguabijia-app-server/.env
# 默认拉昨天;--days 3 回补近 3 天(幂等 upsert,应对偶发漏跑 + 穿山甲对历史数据订正,重跑无害)。
ExecStart=/opt/shaguabijia-app-server/.venv/bin/python -m scripts.sync_pangle_revenue --days 3
SyslogIdentifier=pangle-revenue
# 仅几个 HTTP 请求 + 小批量入库,通常数秒;给 10min 硬超时防穿山甲接口卡死。
TimeoutStartSec=600
# 与主服务 shaguabijia-app-server.service 同款加固。
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ReadWritePaths=/opt/shaguabijia-app-server
ProtectHome=true
+14
View File
@@ -0,0 +1,14 @@
# 每天 10:30 触发一次穿山甲 GroMore T+1 收益拉取入库(Linux 服务器用)。
# 见 pangle-revenue.service 顶部注释的部署步骤。
[Unit]
Description=Run Pangle GroMore daily revenue sync at 10:30
[Timer]
# 穿山甲 T+1、次日约 10:00 出数;10:30 触发留 30min 余量。要错开整点扎堆可微调到 10:35。
OnCalendar=*-*-* 10:30:00
# 服务器宕机/重启后,补跑错过的那一轮(而不是干等次日);叠加 --days 3 回补,漏一两天能自愈。
Persistent=true
AccuracySec=1min
[Install]
WantedBy=timers.target
+9 -3
View File
@@ -46,7 +46,10 @@
| `total` | int | 当前筛选下的**分页总条数**(全量,不受分页影响;= 前端分页器 total) |
| `truncated` | bool | 当前页之后是否还有更多事件(`len(events) > offset + limit`) |
| `total_impressions` | int | 全量展示条数合计 |
| `total_revenue_yuan` | float | 全量收益合计(元) |
| `total_revenue_yuan` | float | 全量**客户端预估**收益合计(元;eCPM 折算) |
| `total_pangle_revenue_yuan` | float \| null | 全量**穿山甲后台预估**收益合计(元;GroMore `revenue`)。穿山甲无用户/类型/场景维度,**仅全量视图**(未按 `user_id`/`ad_type`/`feed_scene` 过滤)有值,否则 `null` |
| `total_pangle_api_revenue_yuan` | float \| null | 全量**穿山甲收益Api**合计(元;GroMore `api_revenue`,各 ADN 回传更接近结算);未配 Reporting / 查当天 / 非全量视图为 `null` |
| `pangle_revenue_available` | bool | 本次是否带穿山甲后台收益(=全量视图且对应日期已同步到 `ad_pangle_daily_revenue`) |
| `total_expected_coin` | int | 全量应发金币合计 |
| `total_actual_coin` | int | 全量实发金币合计 |
| `mismatch_count` | int | 应发≠实发的组数(=0 说明全部按公式发放) |
@@ -57,7 +60,9 @@
|---|---|---|
| `date` | string | 北京时间 `YYYY-MM-DD` |
| `impressions` | int | 当天展示条数合计 |
| `revenue_yuan` | float | 当天预估收益合计(元) |
| `revenue_yuan` | float | 当天客户端预估收益合计(元;eCPM 折算) |
| `pangle_revenue_yuan` | float \| null | 当天穿山甲后台预估收益(元;GroMore `revenue`);非全量视图 / 无数据为 `null` |
| `pangle_api_revenue_yuan` | float \| null | 当天穿山甲收益Api(元;GroMore `api_revenue`);未配 / 当天 / 无数据为 `null` |
| `expected_coin` | int | 当天应发金币合计 |
| `actual_coin` | int | 当天实发金币合计 |
@@ -125,5 +130,6 @@
- **展示 vs 发奖分离**:信息流轮播一会话可展示多条(都计入 `impressions`),但发奖仍按现规则(一会话发一次),`coin` 不因展示条数变化——这是有意设计(用户中途关只记展示不发奖)。
- **历史 Draw 不可拆**:迁移(Draw→普通信息流)前,Draw 发奖混在 `ad_feed_reward_record` 且无类型标记,金币侧统一记 `feed`;迁移后 Draw 不再产生新数据。展示侧 `ad_type` 由客户端上报区分,故 `draw` 桶基本为空。
- **来源字段从上线起齐全**:`app_env`/`our_code_id` 是本期新增列,历史记录为 NULL(报表来源列留空)。
- **收益是预估**:基于客户端上报的 eCPM,非穿山甲后台结算值;以后台报表为结算权威
- **逐条/明细的收益是预估**:`items[].revenue_yuan` 基于客户端上报的 eCPM 折算,非穿山甲后台结算值。
- **穿山甲后台收益(汇总/趋势级)**:`total_pangle_revenue_yuan`(预估 `revenue`)与 `total_pangle_api_revenue_yuan`(收益Api `api_revenue`,更接近结算)来自穿山甲 **GroMore 数据 API**(`integrations/pangle_report` + `scripts/sync_pangle_revenue` 按天 T+1 拉取入 [ad_pangle_daily_revenue](../database/ad_pangle_daily_revenue.md))。穿山甲**不提供分用户/设备/类型/场景维度**(官方明确),最细到 日期×应用×代码位,故只用于汇总与按天趋势的对照,**不挂到逐条事件行**;且仅在全量视图(未按 user/类型/场景过滤)展示。配置见 `.env``PANGLE_REPORT_*`
- **对账聚合级 + 逐条下钻**:行级 `matched` 给出该组(用户×类型×应用×代码位)应发是否==实发;**展开 `records` 即可看该组逐条明细**(eCPM/因子1/份数/LT/因子2/应发/实发/一致)定位到具体记录。独立逐条审计接口 [admin-ad-coin-audit](./admin-ad-coin-audit.md) 仍保留(同一复算口径,可全局按场景/只看不符筛选)。
+45
View File
@@ -0,0 +1,45 @@
# ad_pangle_daily_revenue — 穿山甲 GroMore 天级收益
> 模型:[app/models/ad_pangle_revenue.py](../../app/models/ad_pangle_revenue.py) | 读写:[app/repositories/ad_pangle_revenue.py](../../app/repositories/ad_pangle_revenue.py) | 拉取:[app/integrations/pangle_report.py](../../app/integrations/pangle_report.py) + [scripts/sync_pangle_revenue.py](../../scripts/sync_pangle_revenue.py)
存放从穿山甲 **GroMore 数据 API**(聚合数据报告 API,天级、T+1)按天拉取的收益,供 admin
[广告收益报表](../api/admin-ad-revenue-report.md) 的「穿山甲后台收益」做汇总/趋势级展示,
与客户端自报 eCPM 折算的预估互为对照(看 gap)。
**粒度 = 日期 × 应用(app_env) × 代码位(our_code_id) × 广告源(adn)**。穿山甲**不提供分用户/设备维度**
(官方明确),故本表无 `user_id`,也无法挂到逐条广告事件;报表逐条行仍用客户端预估。
## 数据流
- **写**:`scripts/sync_pangle_revenue`(线上每天 ~10:30 由 systemd timer 跑)调
`pangle_report.fetch_daily_report` 拉昨天(可 `--days N` 回补),维度 `date,site_id,ad_unit_id`,
映射 `site_id→app_env``ad_unit_id→our_code_id``upsert_daily_rows` 落库。按唯一键幂等,T+1 订正可重跑。
⚠️ 用 `ad_unit_id`(广告位ID=104xxx)而非 `code_id`:实测 `code_id` 返回底层各 ADN 代码位(如 `983674557`),对不上我们的口径。
- **读**:`ad_pangle_revenue.aggregate_by_date` 按日期汇总 → admin 报表
`total_pangle_revenue_yuan` / `total_pangle_api_revenue_yuan` / `daily[].pangle_*`
## 字段
| 列 | 类型 | 说明 |
|---|---|---|
| `id` | int PK | |
| `report_date` | str(10) idx | 北京时间 `YYYY-MM-DD`(拉取 `time_zone=8`),与 `ad_ecpm_record.report_date` 同口径可 join |
| `app_env` | str(16) | `prod`(傻瓜比价正式)/ `test`(测试);由 `site_id``settings.pangle_report_site_id_to_env` 映射;未映射记 `site:<id>` |
| `site_id` | str(32) \| null | 原始 GroMore AppId(留痕) |
| `our_code_id` | str(64) idx | 广告位 ID(= 接口 `ad_unit_id` = 客户端配的 104xxx = `ad_ecpm_record.our_code_id`),join key。⚠️ **非** `code_id`(那是底层各 ADN 代码位,对不上);`-1` 为 GroMore 未归因桶 |
| `adn` | str(16) | 广告源(接口 `network`);当前默认口径不分广告源(按 `ad_unit_id` 汇总),统一 `""` |
| `revenue_yuan` | float | 预估收益(元)← 接口 `revenue`(排序价×展示/1000,所有 ADN) |
| `api_revenue_yuan` | float \| null | 收益Api(元)← 接口 `api_revenue`(各 ADN 经 Reporting 回传、更接近结算);未配 / 当天为 `null` |
| `ecpm` | str(32) \| null | 预估 eCPM 原值(接口 `ecpm`,单位**元/千次**,与客户端 getEcpm 的「分」不同) |
| `impressions` | int | 展示次数 ← 接口 `imp_cnt` |
| `currency` | str(8) | 货币(接口 `currency`,正常 `cny`) |
| `synced_at` | datetime | 最近同步写入时间(可被回补覆盖) |
唯一约束 `uq_ad_pangle_daily (report_date, app_env, our_code_id, adn)`;`adn``""` 而非
NULL,避免 NULL 在唯一约束里互不相同导致 upsert 重复。
## 局限
- **T+1**:当天数据穿山甲不出 `api_revenue`,`revenue` 也次日才稳;报表「今天」此两列多为空。
- **`revenue` vs `api_revenue`**:前者所有 ADN 都有(预估口径);后者需后台为该 ADN 配置 Reporting
才回传,部分 ADN 可能长期为空。结算仍以各 ADN 结算单 / 穿山甲后台为准。
@@ -37,7 +37,7 @@ B 安装并首启 App
└─ POST /api/v1/invite/bind { invite_code, channel="clipboard" }
后端 repositories/invite.py bind()
└─ 过四道防线 → 建 invite_relation + 给 A、B 各发金币(同事务原子提交)
└─ 过四道防线 → 建 invite_relation(邀请金币已下线,不写金币流水)
```
手动填码这条:B 在邀请页输码 → `InviteRepository.bindManual()``POST /bind { channel="manual" }` → 同一个 `bind()`
@@ -52,20 +52,20 @@ B 安装并首启 App
|---|---|
| 端点 | `app/api/v1/invite.py`:`GET /api/v1/invite/me`(返回 `invite_code` + `share_url` + 战绩)、`POST /api/v1/invite/bind`(绑定,`channel` = `clipboard` / `manual`)。**均需 Bearer 鉴权**。 |
| share_url 构造 | `invite.py``my_invite`:`settings.INVITE_LANDING_URL + "?ref=" + code``INVITE_LANDING_URL``app/core/config.py`(默认 `https://app-api.shaguabijia.com/media/dl.html`)。 |
| 业务逻辑 | `app/repositories/invite.py`:`ensure_code`(懒生成 6 位邀请码,去混淆字符集,唯一约束碰撞则换码)/ `resolve_inviter`(邀请码→邀请人,大小写不敏感)/ `bind`(下面详述)/ `get_stats`(已邀人数 + 累计金币)。 |
| 业务逻辑 | `app/repositories/invite.py`:`ensure_code`(懒生成 6 位邀请码,去混淆字符集,唯一约束碰撞则换码)/ `resolve_inviter`(邀请码→邀请人,大小写不敏感)/ `bind`(下面详述)/ `get_stats`(已邀人数 + 兼容累计金币字段,当前恒为 0)。 |
| 数据模型 | `app/models/invite.py``InviteRelation`(`inviter_user_id` / `invitee_user_id` / `channel` / `status` / `inviter_coin` / `invitee_coin` / `created_at`)+ `app/models/user.py``User.invite_code` 列。 |
| 迁移 | `alembic/versions/invite_code_and_relation.py`:给 `user``invite_code`(唯一索引)+ 建 `invite_relation` 表。`down_revision = 11a1d08c6f55`。 |
| 收发模型 | `app/schemas/invite.py`:`InviteInfoOut` / `BindInviteIn` / `BindInviteOut`。 |
| 奖励常量 | `app/core/rewards.py`:`INVITE_INVITER_COINS` / `INVITE_INVITEE_COINS`(各 10000 = 1 元)、`INVITE_NEW_USER_WINDOW_HOURS`(72)。 |
| 新人窗口 | `app/core/rewards.py`:`INVITE_NEW_USER_WINDOW_HOURS`(72)。邀请金币已下线,不再配置邀请金币常量。 |
**`bind()` 的四道防线(防重复 / 防刷,看 `repositories/invite.py`):**
1. **被邀请人唯一**:`invitee_user_id` 唯一约束 → 一个 B 只能被绑一次(幂等键,重复返回 `already_bound`,不重复发奖)。
1. **被邀请人唯一**:`invitee_user_id` 唯一约束 → 一个 B 只能被绑一次(幂等键,重复返回 `already_bound`,不重复绑定)。
2. **自邀屏蔽**:`inviter == invitee``self_invite`
3. **新人闸**:`_is_new_user`(B 的 `created_at``INVITE_NEW_USER_WINDOW_HOURS` = 72h 内)才发奖,挡存量老用户互相填码薅羊毛 → 否则 `not_eligible`
3. **新人闸**:`_is_new_user`(B 的 `created_at``INVITE_NEW_USER_WINDOW_HOURS` = 72h 内)才生效,挡存量老用户互相填码刷关系 → 否则 `not_eligible`
4. **手机号唯一**(天然限量):每个 B = 一个真实手机号账号。
发金币复用 `repositories/wallet.py``grant_coins`,与建关系记录在**同一事务**提交,保证"建关系 + 双方加金币"原子
邀请金币已下线:`bind()` 只记录绑定关系,不再写 `coin_transaction`;响应里的金币字段保留兼容旧客户端,当前恒为 0
### 3.2 前端(shaguabijia-app-android)
@@ -120,7 +120,7 @@ B 安装并首启 App
### 4.4 测试硬约束 / 坑(都是机制,不是 bug)
- **B 必须用新手机号**:`invitee_user_id` 唯一,一个 B 只能绑一次;反复测要换号(或手删 `invite_relation` 那行 + 回滚金币)。
- **72h 新人闸**:B 注册后 72 小时内绑才发奖(刚注册肯定满足)。
- **72h 新人闸**:B 注册后 72 小时内绑定才生效(刚注册肯定满足)。
- **A ≠ B**:自邀被屏蔽。
- **B 从点下载到首启 App 之间别复制别的东西**:剪贴板会被覆盖 → 归因丢(剪贴板 deferred deeplink 的固有脆弱性)。
- **笔记本 IP 别变**:debug 包把 `BASE_URL` 的 IP 烧死在编译期,DHCP 一换就连不上 → 给笔记本固定个 LAN IP。
+7771
View File
File diff suppressed because it is too large Load Diff
+52
View File
@@ -0,0 +1,52 @@
/**
* SGApi H5 app-server 后端的薄封装
*
* 同源H5 app-server /media/h5/ 托管后端在 /api/v1 host:port 用相对路径 CORS
* 鉴权JWT Bearertoken SGBridge.getToken() 从原生取(原生持登录态)浏览器调试走 bridge mock token
* 401交原生拉登录(requestLogin)兜底本次请求按失败 reject正式的 refresh 重试策略阶段2 再补
*
* 依赖 shared/bridge.js 先加载( token)
*/
(function (global) {
'use strict';
var BASE = '/api/v1';
function authHeaders() {
var t = (global.SGBridge && global.SGBridge.getToken()) || '';
var h = { 'Content-Type': 'application/json' };
if (t) h['Authorization'] = 'Bearer ' + t;
return h;
}
function handle(res) {
if (res.status === 401) {
// 未授权:拉原生登录(异步),本次请求按失败处理,调用方自行决定是否重试
if (global.SGBridge) global.SGBridge.requestLogin();
return Promise.reject(new Error('unauthorized'));
}
if (!res.ok) {
return res.text().then(function (t) {
return Promise.reject(new Error('http ' + res.status + ' ' + t));
});
}
// 204 / 空体兜底
return res.text().then(function (t) { return t ? JSON.parse(t) : null; });
}
/** GET /api/v1<path>。path 以 / 开头,如 '/savings/battle'。 */
function apiGet(path) {
return fetch(BASE + path, { method: 'GET', headers: authHeaders() }).then(handle);
}
/** POST /api/v1<path>body 自动 JSON 序列化。 */
function apiPost(path, body) {
return fetch(BASE + path, {
method: 'POST',
headers: authHeaders(),
body: JSON.stringify(body || {}),
}).then(handle);
}
global.SGApi = { base: BASE, get: apiGet, post: apiPost };
})(window);
+162
View File
@@ -0,0 +1,162 @@
/**
* SGBridge 傻瓜比价 H5 Android 原生 的桥
*
* 背景四个主 tab(首页/福利/记录/我的)由原生 Compose 改造为 WebView 加载本工程 H5
* H5 只负责"画 + 取后端数据"凡需要原生能力(登录态 / 跳转 / 跳外卖 App / 比价领券 /
* 权限 / 定位 / Toast / 激励视频)一律经本桥调用原生
*
* 协议两个方向
* H5 原生Android WebView.addJavascriptInterface(obj, "SGBridgeNative")
* obj 方法都是同步查询类返回 String(JSON 或纯串)动作类无返回
* 原生 H5原生执行 evaluateJavascript("window.SGBridge._emit('<event>', '<json>')")
* 事件onAuthChange(登录态变) / onBalanceChange(余额变) / onSigninChange(签到态变) / onResume(回前台刷新)
*
* 离线兜底浏览器里( SGBridgeNative) MOCK便于不装 App 直接在本地 server 调样式 / 渲染
* 与原生实现对应 shaguabijia-app-android SGBridge.kt(方法名逐个对齐本文件)
*/
(function (global) {
'use strict';
var native = global.SGBridgeNative || null;
var hasNative = !!native;
// ---- 离线 MOCK(仅无原生时生效,便于浏览器调试渲染;真机一律走 native) ----
var MOCK = {
authState: { loggedIn: true, userId: 1, nickname: '冰', avatarUrl: '', phone: '188****8888' },
token: 'mock-token-for-browser-debug',
deviceId: 'browser-debug-device',
appVersion: '0.0.0-debug',
};
function safeParse(s, fallback) {
try { return s ? JSON.parse(s) : fallback; } catch (e) { return fallback; }
}
// ====== 查询类(同步返回) ======
/** 当前登录态 + 用户基本信息 → {loggedIn, userId, nickname, avatarUrl, phone}。 */
function getAuthState() {
if (hasNative && native.getAuthState) return safeParse(native.getAuthState(), { loggedIn: false });
return MOCK.authState;
}
/** 后端鉴权用的 access token(空串=未登录)。原生持登录态,H5 调后端前取它拼 Bearer。 */
function getToken() {
if (hasNative && native.getToken) return native.getToken() || '';
return MOCK.token;
}
/** 设备唯一标识(心跳 / 领券状态查询等用)。 */
function getDeviceId() {
if (hasNative && native.getDeviceId) return native.getDeviceId() || '';
return MOCK.deviceId;
}
/** App 版本号。 */
function getAppVersion() {
if (hasNative && native.getAppVersion) return native.getAppVersion() || '';
return MOCK.appVersion;
}
/** 已安装的目标电商/外卖 App 包名数组(原生 InstalledApps 探测)。H5 选平台弹窗据此判真实装机态。 */
function getInstalledApps() {
if (hasNative && native.getInstalledApps) return safeParse(native.getInstalledApps(), []);
// MOCK(浏览器无原生):给主流已装,便于本地预览选平台弹窗正常显示"有"。
return ['com.sankuai.meituan', 'com.taobao.taobao', 'com.jingdong.app.mall', 'me.ele'];
}
/** 今日是否已领券(置灰「去领取」→「去查看」)。原生读 CompareButtonState(SP 按天);无桥默认 false。 */
function getCouponClaimedToday() {
if (hasNative && native.getCouponClaimedToday) return !!native.getCouponClaimedToday();
return false;
}
// ====== 动作类(无返回;异步结果走事件) ======
/** 跳原生页。route 取值对齐安卓 Routes(invite / settings / feedback / withdrawal / compareRecords / reportFlow / guideVideo / compareResult / coinHistory / cashHistory / welfareRules ...)。 */
function navigate(route) {
if (hasNative && native.navigate) native.navigate(route);
else console.log('[SGBridge mock] navigate →', route);
}
/** 拉起极光一键登录。结果异步经 onAuthChange 事件回来(不在此函数返回)。 */
function requestLogin() {
if (hasNative && native.requestLogin) native.requestLogin();
else console.log('[SGBridge mock] requestLogin');
}
/** 原生居中 Toast。 */
function toast(msg) {
if (hasNative && native.toast) native.toast(String(msg));
else console.log('[SGBridge mock] toast →', msg);
}
/** 跳美团/外卖 App(deeplink 优先;空则原生按包名启动,未装可跳应用商店)。 */
function openMeituan(deeplink) {
if (hasNative && native.openMeituan) native.openMeituan(deeplink || '');
else console.log('[SGBridge mock] openMeituan →', deeplink);
}
/** 触发 agent 比价流程(原生起无障碍引擎)。 */
function startCompare() {
if (hasNative && native.startCompare) native.startCompare();
else console.log('[SGBridge mock] startCompare');
}
/** 触发一键领券(原生先校验悬浮窗/无障碍权限,再起前台服务)。platforms: string[]。 */
function startCouponClaim(platforms) {
var json = JSON.stringify(platforms || []);
if (hasNative && native.startCouponClaim) native.startCouponClaim(json);
else console.log('[SGBridge mock] startCouponClaim →', json);
}
/** App HomePicker :
* getLaunchIntentForPackage 的拉起(NEW_TASK|CLEAR_TASK 冷启到平台首页)packages: string[](一个平台一组候选包,任一可拉即拉) */
function launchApp(packages) {
var json = JSON.stringify(packages || []);
if (hasNative && native.launchApp) native.launchApp(json);
else console.log('[SGBridge mock] launchApp →', json);
}
// ====== 原生 → H5 事件总线 ======
var listeners = {}; // event → [fn]
/** 订阅原生事件。返回取消订阅函数。 */
function on(event, fn) {
(listeners[event] || (listeners[event] = [])).push(fn);
return function off() {
listeners[event] = (listeners[event] || []).filter(function (f) { return f !== fn; });
};
}
/** 供原生回调:window.SGBridge._emit('onAuthChange', '{...}')。payload 可为 JSON 串或对象。 */
function _emit(event, payload) {
var data = typeof payload === 'string' ? safeParse(payload, payload) : payload;
(listeners[event] || []).forEach(function (fn) {
try { fn(data); } catch (e) { console.error('[SGBridge] listener error', event, e); }
});
}
global.SGBridge = {
hasNative: hasNative,
// 查询
getAuthState: getAuthState,
getToken: getToken,
getDeviceId: getDeviceId,
getAppVersion: getAppVersion,
getInstalledApps: getInstalledApps,
getCouponClaimedToday: getCouponClaimedToday,
// 动作
navigate: navigate,
requestLogin: requestLogin,
toast: toast,
openMeituan: openMeituan,
startCompare: startCompare,
startCouponClaim: startCouponClaim,
launchApp: launchApp,
// 事件
on: on,
_emit: _emit,
};
})(window);
+45
View File
@@ -0,0 +1,45 @@
#!/usr/bin/env bash
# 编 release 正式包并发布到官网下载位 data/media/shaguabijia.apk。
#
# 背景:落地页 dl.html「官网下载」按钮指向 /media/shaguabijia.apk(见 app/main.py download_apk 路由)。
# 该 apk 不入 git(.gitignore 忽略 *.apk),也没有 CI 自动放 —— 以前全靠手动 cp,常忘/放成 debug 包。
# 本脚本把"编 release + 放到位"固化成一条命令。发版时跑一次即可。
#
# 用法: bash scripts/publish_apk.sh
# 产物: <app-server>/data/media/shaguabijia.apk(release 签名、指向生产后端的正式包)
#
# 注意:① 需要 app/jishisongfu-release.jks 存在(release 签名,仅 CTO 持有);缺失会编译失败。
# ② 服务器部署不在本脚本职责内 —— 它只把 apk 放到本仓 data/media/;上线由部署流程把
# data/media/ 同步到生产(或 nginx serve 同目录)。
set -euo pipefail
# 路径全部相对脚本位置算,跟 CWD 无关。
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SERVER_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
ANDROID_DIR="$(cd "$SERVER_DIR/../shaguabijia-app-android" && pwd)"
DEST="$SERVER_DIR/data/media/shaguabijia.apk"
APK_OUT="$ANDROID_DIR/app/build/outputs/apk/release/app-release.apk"
# Android Studio 自带 JBR(本机构建环境约定)。已设 JAVA_HOME 则尊重现有值。
export JAVA_HOME="${JAVA_HOME:-D:/android-studio/jbr}"
echo "[publish_apk] Android 工程: $ANDROID_DIR"
if [ ! -f "$ANDROID_DIR/app/jishisongfu-release.jks" ]; then
echo "[publish_apk] ✗ 缺 app/jishisongfu-release.jks(release 签名),无法出正式包。" >&2
exit 1
fi
echo "[publish_apk] 编 release 包(assembleRelease,R8+release 签名,BASE_URL=生产)…"
( cd "$ANDROID_DIR" && ./gradlew :app:assembleRelease )
if [ ! -f "$APK_OUT" ]; then
echo "[publish_apk] ✗ 没找到产物 $APK_OUT" >&2
exit 1
fi
mkdir -p "$(dirname "$DEST")"
cp "$APK_OUT" "$DEST"
SIZE_MB=$(( $(wc -c < "$DEST") / 1024 / 1024 ))
echo "[publish_apk] ✓ 已发布 → $DEST (${SIZE_MB}MB)"
echo "[publish_apk] 官网下载链接(生产):https://app-api.shaguabijia.com/media/shaguabijia.apk"
echo "[publish_apk] 部署:把 data/media/ 同步到生产服务器(本脚本不负责上线)。"
+151
View File
@@ -0,0 +1,151 @@
"""每日拉取穿山甲 GroMore 天级收益报表入库(供 admin 广告收益报表的「穿山甲后台收益」)。
GroMore 数据 API T+1:次日穿山甲约 10:00 出数建议线上每天 ~10:30 systemd timer 跑一次
(默认拉昨天);穿山甲对历史数据可能订正,故支持回补近 N (幂等 upsert,重跑无害)
用法:
python -m scripts.sync_pangle_revenue # 拉昨天(北京时间)
python -m scripts.sync_pangle_revenue --days 3 # 回补昨天起往前 3 天
python -m scripts.sync_pangle_revenue --date 2026-06-27 # 指定单天
python -m scripts.sync_pangle_revenue --start 2026-06-01 --end 2026-06-27 # 指定区间
约束(来自官方文档):天级跨度 1 个月不早于 12 个月;今天今天以前必须分开查,
本脚本默认只拉昨天及更早,不混查今天未配置 PANGLE_REPORT_* 时直接 no-op 退出
"""
from __future__ import annotations
import argparse
import sys
from datetime import date, timedelta
# Windows 控制台按 UTF-8 输出中文/¥
try:
sys.stdout.reconfigure(encoding="utf-8") # type: ignore[attr-defined]
except Exception: # noqa: BLE001
pass
from app.core.config import settings
from app.core.rewards import cn_today
from app.db.session import SessionLocal
from app.integrations import pangle_report
from app.integrations.pangle_report import PangleReportError
from app.repositories import ad_pangle_revenue as repo
# 拉取维度固定:date×site_id×ad_unit_id。⚠️ 用 ad_unit_id(GroMore 广告位ID = 我们客户端配的 104xxx
# = ad_ecpm_record.our_code_id)而**非** code_id —— 实测 code_id 返回的是底层各 ADN 的代码位
# (如 983674557 / youlianghui_xinxiliu),对不上我们的口径。指标取预估 + Api 两个收益。
_DIMENSIONS = "date,site_id,ad_unit_id"
_METRICS = "revenue,api_revenue,ecpm,imp_cnt"
def _to_float(v: object) -> float | None:
"""接口数值字段都是字符串;空/非法 → None(api_revenue 缺失时即为 None,前端显示「-」)。"""
if v is None or str(v).strip() == "":
return None
try:
return float(str(v).strip())
except (TypeError, ValueError):
return None
def _row_date(row: dict) -> str | None:
"""报表行的日期:优先 date 维度,回退 start_date;取前 10 位 'YYYY-MM-DD'"""
raw = row.get("date") or row.get("start_date") or ""
raw = str(raw).strip()
return raw[:10] if len(raw) >= 10 else None
def _normalize(rows: list[dict]) -> tuple[list[dict], int, set[str]]:
"""接口原始行 → 入库行;返回 (入库行, 跳过数, 未映射 site_id 集合)。
site_id settings 映射成 app_env(prod/test);未映射的仍入库(app_env=site:<id>)但提示,
避免静默丢收益 code_id / 日期的行跳过
"""
site_map = settings.pangle_report_site_id_to_env
out: list[dict] = []
skipped = 0
unmapped: set[str] = set()
for row in rows:
rdate = _row_date(row)
# 用 ad_unit_id(GroMore 广告位ID = 我们的 104xxx),不是 code_id(底层 ADN 代码位)
code_id = str(row.get("ad_unit_id") or "").strip()
if not rdate or not code_id:
skipped += 1
continue
site_id = str(row.get("site_id") or "").strip()
app_env = site_map.get(site_id)
if app_env is None:
app_env = f"site:{site_id}" if site_id else "unknown"
if site_id:
unmapped.add(site_id)
out.append({
"report_date": rdate,
"app_env": app_env,
"site_id": site_id or None,
"our_code_id": code_id,
"adn": "",
"revenue_yuan": _to_float(row.get("revenue")) or 0.0,
"api_revenue_yuan": _to_float(row.get("api_revenue")),
"ecpm": (str(row.get("ecpm")).strip() or None) if row.get("ecpm") is not None else None,
"impressions": int(_to_float(row.get("imp_cnt")) or 0),
"currency": "cny",
})
return out, skipped, unmapped
def sync_range(start_date: str, end_date: str) -> None:
if not settings.pangle_report_configured:
print("PANGLE_REPORT_* 未配置,跳过(no-op)。需在后台「接入中心 → GroMore-API」领取 "
"user_id/role_id/Security Key 并填入 .env。")
return
print(f"拉取 GroMore 天级收益 {start_date} ~ {end_date} (dims={_DIMENSIONS}) ...")
try:
raw = pangle_report.fetch_daily_report(
start_date=start_date, end_date=end_date,
dimensions=_DIMENSIONS, metrics=_METRICS, time_zone=8,
)
except PangleReportError as e:
print(f"❌ 拉取失败:{e}")
sys.exit(1)
rows, skipped, unmapped = _normalize(raw)
if unmapped:
print(f"⚠️ 有未映射 site_id(收益仍入库,app_env=site:<id>):{sorted(unmapped)};"
f"如需归入 prod/test,补 PANGLE_REPORT_SITE_ID_* 配置。")
with SessionLocal() as db:
stats = repo.upsert_daily_rows(db, rows)
total_rev = round(sum(r["revenue_yuan"] for r in rows), 4)
print(f"✅ 完成:接口 {len(raw)} 行 → 入库 {len(rows)} 行(跳过 {skipped}),"
f"新增 {stats['inserted']} / 更新 {stats['updated']};预估收益合计 ¥{total_rev}")
def main() -> None:
ap = argparse.ArgumentParser(description="拉取穿山甲 GroMore 天级收益报表入库")
ap.add_argument("--date", help="指定单天 YYYY-MM-DD")
ap.add_argument("--start", help="区间起始 YYYY-MM-DD(与 --end 同用)")
ap.add_argument("--end", help="区间结束 YYYY-MM-DD(与 --start 同用)")
ap.add_argument("--days", type=int, help="回补:昨天起往前 N 天(含昨天)")
args = ap.parse_args()
yesterday = cn_today() - timedelta(days=1)
if args.date:
start = end = args.date
elif args.start and args.end:
start, end = args.start, args.end
elif args.days:
start = (yesterday - timedelta(days=args.days - 1)).isoformat()
end = yesterday.isoformat()
else:
start = end = yesterday.isoformat()
# 防呆:跨度 > 31 天直接拒(接口上限 1 个月,超了会返回 114)
if (date.fromisoformat(end) - date.fromisoformat(start)).days > 31:
print("❌ 区间跨度超过 31 天(接口上限 1 个月),请分批拉。")
sys.exit(1)
sync_range(start, end)
if __name__ == "__main__":
main()
+2 -1
View File
@@ -305,11 +305,12 @@ def test_feed_reward_grants_by_10_second_units(client) -> None:
"duration_seconds": 30,
"adn": "pangle",
"slot_id": "slot_feed",
"display_coin": 4,
}
r = client.post("/api/v1/ad/feed-reward", json=payload, headers=_auth(token))
assert r.status_code == 200, r.text
body = r.json()
expected = sum(calculate_ad_reward_coin("200", i) for i in range(1, 4))
expected = 4
assert body["granted"] is True
assert body["status"] == "granted"
assert body["unit_count"] == 3
+3 -1
View File
@@ -64,7 +64,9 @@ def test_dashboard_overview(admin_client: TestClient, admin_token: str) -> None:
assert data["users"]["total"] >= 1
assert data["coins"]["granted_total"] >= 5000
assert "success_rate" in data["comparison"]
assert data["cps"]["available"] is False
assert data["cps"]["available"] is True
assert "meituan_order_count" in data["cps"]
assert "jd_order_count" in data["cps"]
def test_user_list_and_detail(admin_client: TestClient, admin_token: str) -> None:
+3
View File
@@ -44,6 +44,7 @@ def _food_payload(trace_id: str) -> dict:
"skipped_dish_names": ["黑牛肉卷"],
"total_dish_count": 3,
"information": "在美团找到同店,到手价 ¥123.50",
"total_ms": 12345,
}
@@ -71,6 +72,8 @@ def test_report_and_derive(client) -> None:
assert d["information"] == "在美团找到同店,到手价 ¥123.50"
assert d["store_name"] == "海底捞(朝阳店)"
assert d["total_dish_count"] == 3
assert d["total_ms"] == 12345
assert d["raw_payload"]["total_ms"] == 12345
assert d["skipped_dish_count"] == 1
assert d["skipped_dish_names"] == ["黑牛肉卷"]
assert len(d["comparison_results"]) == 3
+71
View File
@@ -11,6 +11,7 @@ from app.admin.repositories import admin_user as admin_repo
from app.admin.repositories import cps as cps_repo
from app.db.session import SessionLocal
from app.models.cps_link import CpsClick
from app.models.cps_order import CpsOrder
from app.models.cps_wx_user import CpsWxUser
from app.repositories import cps_link as cps_link_repo
@@ -180,3 +181,73 @@ def test_day_users_cross_year(admin_client: TestClient, admin_token: str) -> Non
assert len(users) == 1
assert users[0]["openid"] == openid
assert users[0]["visit_count"] == 5 # 次年 01-01 那条被时间窗排除
def test_jd_reconcile_updates_dashboard(
admin_client: TestClient, admin_token: str, monkeypatch: pytest.MonkeyPatch
) -> None:
"""京东拉单:按订单行入库;大盘只统计有效 validCode 的佣金。"""
def fake_query_order_rows(**kwargs):
start = kwargs["start_time"].astimezone(_BJ)
if start.hour != 10 or kwargs["page_index"] != 1:
return {"rows": [], "has_more": False}
return {
"rows": [
{
"id": "pytest-jd-row-valid",
"orderId": "pytest-jd-order-1",
"skuId": "sku-1",
"skuName": "京东测试商品",
"orderTime": "2026-06-25 10:10:00",
"modifyTime": "2026-06-25 10:20:00",
"validCode": "16",
"estimateCosPrice": "19.90",
"estimateFee": "1.23",
"actualFee": "2.34",
"commissionRate": "10.00",
},
{
"id": "pytest-jd-row-invalid",
"orderId": "pytest-jd-order-2",
"skuId": "sku-2",
"skuName": "京东无效订单",
"orderTime": "2026-06-25 10:15:00",
"modifyTime": "2026-06-25 10:25:00",
"validCode": "4",
"estimateCosPrice": "9.90",
"estimateFee": "0.50",
"commissionRate": "5.00",
},
],
"has_more": False,
}
monkeypatch.setattr("app.integrations.jd_union.query_order_rows", fake_query_order_rows)
r = admin_client.post(
"/admin/api/cps/orders/reconcile",
params={
"platform": "jd",
"date_from": "2026-06-25",
"date_to": "2026-06-25",
"query_time_type": 1,
},
headers=_auth(admin_token),
)
assert r.status_code == 200, r.text
assert r.json()["fetched"] == 2
with SessionLocal() as db:
rows = db.query(CpsOrder).filter(CpsOrder.platform == "jd").all()
assert len([o for o in rows if o.order_id.startswith("jd:pytest-jd-row")]) == 2
overview = admin_client.get(
"/admin/api/stats/overview",
params={"date_from": "2026-06-25", "date_to": "2026-06-25"},
headers=_auth(admin_token),
)
assert overview.status_code == 200, overview.text
cps = overview.json()["cps"]
assert cps["jd_order_count"] >= 1
assert cps["jd_commission_cents"] >= 234
assert cps["jd_invalid_count"] >= 1
+134
View File
@@ -0,0 +1,134 @@
"""穿山甲 GroMore 数据 API 客户端测试。
重点锚定**签名算法**:用官方文档 Python / Java 示例里给出的参数 + secure_key sign两个
测试向量做断言,保证我们的 build_sign 与穿山甲服务端逐字节一致(签错=接口直接 101 验签失败)
另测拉取的翻页业务码非 100 抛错响应解析
"""
from __future__ import annotations
import pytest
from app.core.config import settings
from app.integrations import pangle_report
from app.integrations.pangle_report import PangleReportError, build_sign
# ---- 签名:官方文档测试向量(完整 key 与期望 sign 来自文档代码注释)----
def test_build_sign_matches_doc_python_vector():
params = {
"current_time": "2022-01-06 16:10:54",
"sign_type": "MD5",
"version": "2.0",
"user_id": 459,
"role_id": 459,
}
sign = build_sign(params, "7f2ddb34d999ea61c00246ea3971529c")
assert sign == "57b9fed9b6c09cefdced7fa770fe341d"
def test_build_sign_matches_doc_java_vector():
params = {
"user_id": 459,
"role_id": 459,
"version": "2.0",
"sign_type": "MD5",
"current_time": "2022-01-06 16:05:49",
}
sign = build_sign(params, "1c589c70f7bb2746027ce90c33d2544c")
assert sign == "4d47b3a7679448c3da5c1d05e5291533"
def test_build_sign_drops_sign_and_empty_fields():
"""sign 字段与空值字段不参与签名:带上它们应得到与不带时相同的 sign。"""
base = {"version": "2.0", "user_id": 459, "role_id": 459, "sign_type": "MD5"}
with_noise = {**base, "sign": "deadbeef", "code_ids": "", "os": None}
assert build_sign(with_noise, "key123") == build_sign(base, "key123")
# ---- 拉取:伪造 httpx 客户端,验证翻页 / 解析 / 业务码 ----
class _FakeResp:
def __init__(self, payload: dict):
self._payload = payload
def raise_for_status(self) -> None:
pass
def json(self) -> dict:
return self._payload
class _FakeClient:
"""按 offset 返回分页响应;记录每次请求的 params 供断言。"""
def __init__(self, pages: list[dict]):
self._pages = pages
self.calls: list[dict] = []
def __enter__(self):
return self
def __exit__(self, *a):
return False
def get(self, url: str, params: dict):
self.calls.append(params)
# 第 N 次调用取第 N 页(has_next 驱动翻页);超出用最后一页兜底
page = self._pages[min(len(self.calls) - 1, len(self._pages) - 1)]
return _FakeResp(page)
@pytest.fixture
def _configured(monkeypatch):
monkeypatch.setattr(settings, "PANGLE_REPORT_USER_ID", 459)
monkeypatch.setattr(settings, "PANGLE_REPORT_ROLE_ID", 459)
monkeypatch.setattr(settings, "PANGLE_REPORT_SECURITY_KEY", "key123")
def test_fetch_daily_report_paginates_and_parses(monkeypatch, _configured):
page1 = {
"code": "100", "message": "",
"data": {
"has_next": "1", "currency": "cny", "total": 3,
"report_list": [
{"start_date": "2026-06-27", "site_id": "5830519", "ad_unit_id": "104142227",
"revenue": "1.23", "api_revenue": "1.05", "ecpm": "0.80", "imp_cnt": "1537"},
],
},
}
page2 = {
"code": "100", "message": "",
"data": {
"has_next": "0", "currency": "cny", "total": 3,
"report_list": [
{"start_date": "2026-06-27", "site_id": "5832303", "ad_unit_id": "104137445",
"revenue": "0.00", "api_revenue": "", "ecpm": "0", "imp_cnt": "12"},
],
},
}
fake = _FakeClient([page1, page2])
monkeypatch.setattr(pangle_report.httpx, "Client", lambda *a, **k: fake)
rows = pangle_report.fetch_daily_report(start_date="2026-06-27", end_date="2026-06-27")
assert len(rows) == 2 # 两页都被取到(has_next 驱动翻页)
assert rows[0]["ad_unit_id"] == "104142227"
assert rows[1]["api_revenue"] == ""
# 每次请求都带上签名与必填鉴权参数
assert "sign" in fake.calls[0]
assert fake.calls[0]["user_id"] == 459 and fake.calls[0]["version"] == "2.0"
def test_fetch_daily_report_raises_on_business_error(monkeypatch, _configured):
fail = {"code": "118", "message": "no revenue permission", "data": {}}
monkeypatch.setattr(
pangle_report.httpx, "Client", lambda *a, **k: _FakeClient([fail])
)
with pytest.raises(PangleReportError):
pangle_report.fetch_daily_report(start_date="2026-06-27", end_date="2026-06-27")
def test_fetch_daily_report_no_config_raises(monkeypatch):
monkeypatch.setattr(settings, "PANGLE_REPORT_USER_ID", 0)
with pytest.raises(PangleReportError):
pangle_report.fetch_daily_report(start_date="2026-06-27", end_date="2026-06-27")
+2 -1
View File
@@ -52,7 +52,8 @@ def call_raw(path: str, body_obj: dict) -> dict:
}
url = f"{settings.MT_CPS_HOST}{path}"
t0 = time.time()
resp = httpx.post(url, content=body, headers=headers, timeout=settings.MT_CPS_TIMEOUT_SEC)
# trust_env=False: 美团是国内域名,强制直连绕开本机代理(代理会掐断 TLS 握手,报 SSL EOF)
resp = httpx.post(url, content=body, headers=headers, timeout=settings.MT_CPS_TIMEOUT_SEC, trust_env=False)
ms = int((time.time() - t0) * 1000)
try:
j = resp.json()