Compare commits

...

10 Commits

Author SHA1 Message Date
ouzhou 04edb50acc feat(admin/ad): 提现自动对账后台开关 + 客户端广告 flags 端点 + 穿山甲多 m-key 验签 (#59)
- 自动对账接入 app_config 运行时配置(新增 bool 配置类型):env 部署总闸 + DB 运营日常开关
  双层;worker 每轮读 DB 即时生效,健康检查改报「env AND DB」实际生效态
- 新增 GET /api/v1/platform/flags(不鉴权)下发 comparing_ad_enabled 远程 kill-switch,
  客户端拉取后缓存;空库回退默认 True
- 穿山甲发奖回调支持多激励位 m-key(三命名项 PANGLE_REWARD_SECRET_TEST/_DEDICATED/_PROD
  + 旧逗号分隔合并去重),verify_callback_sign_any 逐个验签任一通过即受理;向后兼容旧单 key
- 补 test_platform / bool config 用例;app_config 文档同步

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: OuYingJun1024 <1034284404@qq.com>
Reviewed-on: #59
Co-authored-by: ouzhou <ouzhou@wonderable.ai>
Co-committed-by: ouzhou <ouzhou@wonderable.ai>
2026-06-17 10:37:54 +08:00
marco 3483c1bba9 Merge origin/main (meituan-etl #57/#58) into CPS 分发
整合实习生 meituan-etl 全国扩城 + NUL 清洗,与本次 CPS 分发对账并线。

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 10:03:29 +08:00
marco 277f9b16a2 feat(cps): 群发券分发与对账 + 短链点击追踪
群(sid)/活动池/转链(带sid)/美团 query_order 按 sid 对账/按群统计;
短链 /c/{code} 记点击(PV/UV)→302 跳美团,点击→下单漏斗。

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 10:00:29 +08:00
marco 9d93b70b9b feat(user): username 对外展示 + 默认昵称 + 存量回填
代提工作区既有的他人在制品(非本次 CPS);CPS 迁移链 cps_tables 依赖其
b3f1a2c4d5e6 迁移,需一并提交。

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 10:00:29 +08:00
chenshuobo f853938095 fix(meituan-etl): 清洗文本 NUL 字节 + 逐城入库容错(修全量首灌崩在脏数据) (#58)
全国 359 城全量首灌崩在厦门:美团某券文本字段含 NUL(0x00),PostgreSQL
text/jsonb 拒绝该字节,整批 upsert 抛 DataError → --once 进程崩、后续 300+ 城
全不跑(本地 20 城没撞上、跑全国才暴露)。两处修复:
- _strip_nul 递归清洗入库 dict 所有字符串(含 raw JSON)的 NUL;
- 逐城 _upsert 包 try/except + rollback,单城入库失败记 fails 跳过,不再让
  一条脏数据 / 一次抖动拖垮整轮 359 城。

---------

Co-authored-by: chenshuobo <1119780489@qq.com>
Reviewed-on: #58
Co-authored-by: chenshuobo <chenshuobo@wonderable.ai>
Co-committed-by: chenshuobo <chenshuobo@wonderable.ai>
2026-06-16 21:19:47 +08:00
chenshuobo 783dfd059d feat(meituan-etl): 离线库扩到全国 359 城(多城并发抓取入库) (#57)
智能推荐 / 销量最高两 tab 的离线库此前只有北京(ETL 写死 cityId),改为遍历
美团官方城市字典 359 个地级市全量抓取。实测一个地级市 cityId 已覆盖其下辖县级市
(徐州→邳州/新沂/睢宁等),按地级市抓即可,无需区县层级。

- 城市字典:tools/gen_meituan_cities.py 从美团 Excel 生成随仓库 JSON
  (app/integrations/data/meituan_cities.json,359 城),app/integrations/cities.py 读取;
- ETL:city_id 参数化 + 城市级并发(默认 12)+ worker 启动错峰削平瞬时峰值
  + 主线程逐城串行入库(Session 不跨线程);
- 配速实测:单城全量 ~110s/~2300 条;15 并发抓完一轮 ~50-60min,402 占 3% 退避全消化;
  每 3h 一轮全量(--interval 10800),窗口充裕;
- prune 双护栏:本轮 0 入库 或 失败城占比 >5% 时跳过,防上游故障/大面积限流误删全表;
- 仅写入侧;读取侧(rec/top-sales 按城过滤)待后续(依赖用户定位→cityId 映射,字典无经纬度)。

---------

Co-authored-by: chenshuobo <1119780489@qq.com>
Reviewed-on: #57
Co-authored-by: chenshuobo <chenshuobo@wonderable.ai>
Co-committed-by: chenshuobo <chenshuobo@wonderable.ai>
2026-06-16 15:40:46 +08:00
marco f97048ff56 fix(alembic): 合并多 head (store_mapping_jd_dl_invalid + ad_revenue) 修复 0.1.2 部署
PR #56 的 jd_deeplink_invalid 迁移与 ad_revenue 报表迁移从同一 branchpoint
分叉成双 head, 部署时 alembic upgrade head 报 "Multiple head revisions"。
加一个 no-op merge 节点统一 DAG, 两支迁移列不冲突。

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 01:39:05 +08:00
marco f15ca74a22 feat(store-mapping): JD 缓存 deeplink 失效标记(对标淘宝) (#56)
Reviewed-on: #56
2026-06-16 01:25:57 +08:00
ouzhou f7d86011c1 feat(ad-revenue): admin 广告收益报表(按 用户/日期/类型/应用/代码位 聚合) (#54)
- 新增 GET /admin/api/ad-revenue-report:展示条数/收益 + 复用金币审计逐条复算做发奖对账
- ad_ecpm/ad_reward/ad_feed_reward 各加 app_env + our_code_id 两列(alembic 迁移)
- ecpm-report / feed-reward 接收并落库 app_env/our_code_id;激励发奖按 ad_session_id 回填
- ad_audit 抽出 audit_rows,报表与逐条审计复用同一复算口径
- 组级 matched 改「组内逐条全一致」,避免应发和==实发和的互相抵消掩盖错误
- list_feedbacks 改 offset 分页并返回 total(配合 admin 页码分页)
- 反馈正文上限 _CONTENT_MAX 2000→200
- 文档:新增 admin-ad-revenue-report,更新 ecpm/feed-reward/feedback 及对应 db docs

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: OuYingJun1024 <1034284404@qq.com>
Reviewed-on: #54
Co-authored-by: ouzhou <ouzhou@wonderable.ai>
Co-committed-by: ouzhou <ouzhou@wonderable.ai>
2026-06-15 23:13:14 +08:00
marco 9ec9d2389d feat(store_mapping): 淘宝缓存 deeplink 失效检测→标记→回退(app-server 侧) (#53)
加 taobao_deeplink_invalid_at 列(迁移+model);mark_taobao_deeplink_invalid 按 shopId 标记所有行;lookup_nearest 过滤失效淘宝候选;新增 POST /internal/store-mapping/invalidate。+ 7 单测。

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Reviewed-on: #53
2026-06-15 18:07:32 +08:00
78 changed files with 4943 additions and 149 deletions
+10 -2
View File
@@ -88,9 +88,17 @@ AUTO_EXCHANGE_ENABLED=true
# ===== 穿山甲激励视频(服务端发奖回调)===== # ===== 穿山甲激励视频(服务端发奖回调)=====
# 看完激励视频后穿山甲服务器 S2S 回调本服务发金币(客户端不参与发奖)。 # 看完激励视频后穿山甲服务器 S2S 回调本服务发金币(客户端不参与发奖)。
# PANGLE_REWARD_SECRET 是穿山甲后台配置的"奖励校验密钥",用于验签,从后台取到后填这里; # 穿山甲"奖励校验密钥"(m-key),验签,从 GroMore 后台各广告位取到后填这里
# 配齐并把 ENABLED=true 后,/api/v1/ad/pangle-callback 才受理回调(否则 503)。 # 配齐(任一非空)并把 ENABLED=true 后,/api/v1/ad/pangle-callback 才受理回调(否则 503)。
# 每个激励位 m-key 不同但共用同一回调 URL → 各位分开一行配(留空的忽略);验签逐个试、任一通过即接受。
PANGLE_CALLBACK_ENABLED=false PANGLE_CALLBACK_ENABLED=false
# 测试应用 激励位 104099649
PANGLE_REWARD_SECRET_TEST=
# 测试应用 专属激励位 104127529
PANGLE_REWARD_SECRET_TEST_DEDICATED=
# 正式应用 激励位 104099389
PANGLE_REWARD_SECRET_PROD=
# (旧用法,仍兼容:单个或逗号分隔的多个 m-key,会与上面三个合并去重)
PANGLE_REWARD_SECRET= PANGLE_REWARD_SECRET=
# ⚠️ 仅本地联调:true 时开放 POST /api/v1/ad/test-grant,让 debug 客户端看完广告直接发奖, # ⚠️ 仅本地联调:true 时开放 POST /api/v1/ad/test-grant,让 debug 客户端看完广告直接发奖,
# 验证"看广告→金币到账"全链路(未部署公网、穿山甲 S2S 打不到本地时用)。生产必须 false(绕过反作弊)。 # 验证"看广告→金币到账"全链路(未部署公网、穿山甲 S2S 打不到本地时用)。生产必须 false(绕过反作弊)。
@@ -0,0 +1,26 @@
"""merge store_mapping_jd_dl_invalid and ad_revenue heads (0.1.2)
Revision ID: 45047b5a884c
Revises: d4e68464761d, store_mapping_jd_dl_invalid
Create Date: 2026-06-16 01:38:26.273226
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = '45047b5a884c'
down_revision: Union[str, Sequence[str], None] = ('d4e68464761d', 'store_mapping_jd_dl_invalid')
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
pass
def downgrade() -> None:
pass
@@ -0,0 +1,39 @@
"""ad revenue report columns: app_env + our_code_id
给 ad_ecpm_record / ad_reward_record / ad_feed_reward_record 各加两列:
- app_env:我们的穿山甲应用环境(prod=傻瓜比价正式 / test=测试应用)
- our_code_id:我们在穿山甲后台配置的代码位 ID(104xxx,非底层 mediation rit)
供「广告收益报表」按 用户/日期/广告类型/应用/代码位 聚合 展示条数/收益/金币。
旧数据这两列为 NULL(报表里来源列留空),新数据由客户端上报/发奖时回填。
Revision ID: ad_revenue_report_cols
Revises: coupon_engage_per_package
Create Date: 2026-06-15
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "ad_revenue_report_cols"
down_revision = "coupon_engage_per_package"
branch_labels = None
depends_on = None
_TABLES = ("ad_ecpm_record", "ad_reward_record", "ad_feed_reward_record")
def upgrade() -> None:
for table in _TABLES:
op.add_column(table, sa.Column("app_env", sa.String(length=16), nullable=True))
op.add_column(table, sa.Column("our_code_id", sa.String(length=64), nullable=True))
def downgrade() -> None:
for table in _TABLES:
op.drop_column(table, "our_code_id")
op.drop_column(table, "app_env")
@@ -0,0 +1,79 @@
"""user.username(对外展示账号 ID)+ 默认昵称,并回填存量用户
Revision ID: b3f1a2c4d5e6
Revises: 45047b5a884c
Create Date: 2026-06-16 12:00:00.000000
加 user.username(11 位纯数字、首位非 1 与手机号天然区分、全局唯一)。存量用户:
生成唯一 username,昵称为空的补 9 位字母数字随机昵称。
迁移自包含(不 import app 业务代码,逻辑冻结为当时快照)。
"""
import secrets
import string
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = 'b3f1a2c4d5e6'
down_revision: Union[str, Sequence[str], None] = '45047b5a884c'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
_USERNAME_FIRST = "23456789" # 首位避前导 0、避 1(手机号都以 1 开头)
_USERNAME_DIGITS = "0123456789"
_NICKNAME_ALPHABET = string.ascii_letters + string.digits
def _gen_username() -> str:
return secrets.choice(_USERNAME_FIRST) + "".join(
secrets.choice(_USERNAME_DIGITS) for _ in range(10)
)
def _gen_nickname() -> str:
return "".join(secrets.choice(_NICKNAME_ALPHABET) for _ in range(9))
def upgrade() -> None:
# 1. 先加可空列(存量行此刻还没值)
with op.batch_alter_table('user', schema=None) as batch_op:
batch_op.add_column(sa.Column('username', sa.String(length=11), nullable=True))
# 2. 回填存量:每人一个唯一 username;昵称为空的补随机昵称。
# 表里 username 原本全空,本批用 used 防互撞即可(无既有非空值需规避)。
# "user" 是 PostgreSQL 保留字,必须加双引号(SQLite 也接受双引号标识符)。
bind = op.get_bind()
rows = bind.execute(sa.text('SELECT id, nickname FROM "user"')).fetchall()
used = set()
for row in rows:
uid, nick = row[0], row[1]
while True:
uname = _gen_username()
if uname not in used:
used.add(uname)
break
if nick is None or not str(nick).strip():
bind.execute(
sa.text('UPDATE "user" SET username = :u, nickname = :n WHERE id = :i'),
{"u": uname, "n": _gen_nickname(), "i": uid},
)
else:
bind.execute(
sa.text('UPDATE "user" SET username = :u WHERE id = :i'),
{"u": uname, "i": uid},
)
# 3. 收紧:NOT NULL + 唯一索引(对齐模型 unique=True, index=True)
with op.batch_alter_table('user', schema=None) as batch_op:
batch_op.alter_column('username', existing_type=sa.String(length=11), nullable=False)
batch_op.create_index('ix_user_username', ['username'], unique=True)
def downgrade() -> None:
with op.batch_alter_table('user', schema=None) as batch_op:
batch_op.drop_index('ix_user_username')
batch_op.drop_column('username')
+61
View File
@@ -0,0 +1,61 @@
"""cps_link / cps_click tables (群发短链 + 点击统计)
Revision ID: cps_link_tables
Revises: cps_tables
Create Date: 2026-06-16 13:20:00.000000
群发券链接套一层 /c/{code} 做点击统计:cps_link(短码→美团目标) + cps_click(点击事件)。
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = "cps_link_tables"
down_revision: Union[str, Sequence[str], None] = "cps_tables"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
_NOW = sa.text("CURRENT_TIMESTAMP")
def upgrade() -> None:
op.create_table(
"cps_link",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("code", sa.String(length=16), nullable=False),
sa.Column("group_id", sa.Integer(), nullable=False),
sa.Column("activity_id", sa.Integer(), nullable=False),
sa.Column("sid", sa.String(length=64), nullable=False),
sa.Column("platform", sa.String(length=20), nullable=False, server_default="meituan"),
sa.Column("target_url", sa.String(length=1024), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=_NOW, nullable=False),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("code", name="uq_cps_link_code"),
)
op.create_index("ix_cps_link_code", "cps_link", ["code"])
op.create_index("ix_cps_link_group_id", "cps_link", ["group_id"])
op.create_index("ix_cps_link_activity_id", "cps_link", ["activity_id"])
op.create_index("ix_cps_link_sid", "cps_link", ["sid"])
op.create_table(
"cps_click",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("link_id", sa.Integer(), nullable=False),
sa.Column("group_id", sa.Integer(), nullable=False),
sa.Column("sid", sa.String(length=64), nullable=False),
sa.Column("ip", sa.String(length=64), nullable=True),
sa.Column("ua", sa.String(length=512), nullable=True),
sa.Column("clicked_at", sa.DateTime(timezone=True), server_default=_NOW, nullable=False),
sa.PrimaryKeyConstraint("id"),
)
op.create_index("ix_cps_click_link_id", "cps_click", ["link_id"])
op.create_index("ix_cps_click_group_id", "cps_click", ["group_id"])
op.create_index("ix_cps_click_sid", "cps_click", ["sid"])
op.create_index("ix_cps_click_clicked_at", "cps_click", ["clicked_at"])
def downgrade() -> None:
op.drop_table("cps_click")
op.drop_table("cps_link")
+92
View File
@@ -0,0 +1,92 @@
"""cps_group / cps_activity / cps_order tables (CPS 分发与对账)
Revision ID: cps_tables
Revises: b3f1a2c4d5e6
Create Date: 2026-06-16 12:40:00.000000
群发 CPS 优惠券的分发与对账:cps_group(群=sid) + cps_activity(可推广活动池) +
cps_order(美团 query_order 按 sid 拉回的订单明细)。金额统一存「分」。
order_id 全局唯一(reconcile 按它 upsert,订单状态会变则更新)。
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
from sqlalchemy.dialects import postgresql
revision: str = "cps_tables"
down_revision: Union[str, Sequence[str], None] = "b3f1a2c4d5e6"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
_NOW = sa.text("CURRENT_TIMESTAMP")
def upgrade() -> None:
op.create_table(
"cps_group",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("sid", sa.String(length=64), nullable=False),
sa.Column("name", sa.String(length=128), nullable=False),
sa.Column("member_count", sa.Integer(), nullable=True),
sa.Column("status", sa.String(length=20), nullable=False, server_default="active"),
sa.Column("remark", sa.String(length=256), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=_NOW, nullable=False),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("sid", name="uq_cps_group_sid"),
)
op.create_index("ix_cps_group_sid", "cps_group", ["sid"])
op.create_table(
"cps_activity",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("platform", sa.String(length=20), nullable=False, server_default="meituan"),
sa.Column("name", sa.String(length=128), nullable=False),
sa.Column("act_id", sa.String(length=64), nullable=True),
sa.Column("product_view_sign", sa.String(length=128), nullable=True),
sa.Column("status", sa.String(length=20), nullable=False, server_default="active"),
sa.Column("remark", sa.String(length=256), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=_NOW, nullable=False),
sa.PrimaryKeyConstraint("id"),
)
op.create_table(
"cps_order",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("order_id", sa.String(length=64), nullable=False),
sa.Column("sid", sa.String(length=64), nullable=True),
sa.Column("act_id", sa.String(length=64), nullable=True),
sa.Column("biz_line", sa.Integer(), nullable=True),
sa.Column("trade_type", sa.Integer(), nullable=True),
sa.Column("pay_price_cents", sa.Integer(), nullable=True),
sa.Column("commission_cents", sa.Integer(), nullable=True),
sa.Column("commission_rate", sa.String(length=16), nullable=True),
sa.Column("refund_price_cents", sa.Integer(), nullable=True),
sa.Column("refund_profit_cents", sa.Integer(), nullable=True),
sa.Column("mt_status", sa.String(length=8), nullable=True),
sa.Column("invalid_reason", sa.String(length=128), nullable=True),
sa.Column("product_name", sa.String(length=512), nullable=True),
sa.Column("pay_time", sa.DateTime(timezone=True), nullable=True),
sa.Column("mt_update_time", sa.DateTime(timezone=True), nullable=True),
sa.Column(
"raw",
sa.JSON().with_variant(postgresql.JSONB(), "postgresql"),
nullable=False,
),
sa.Column("first_seen", sa.DateTime(timezone=True), server_default=_NOW, nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), server_default=_NOW, nullable=False),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("order_id", name="uq_cps_order_order_id"),
)
op.create_index("ix_cps_order_order_id", "cps_order", ["order_id"])
op.create_index("ix_cps_order_sid", "cps_order", ["sid"])
op.create_index("ix_cps_order_act_id", "cps_order", ["act_id"])
op.create_index("ix_cps_order_mt_status", "cps_order", ["mt_status"])
op.create_index("ix_cps_order_pay_time", "cps_order", ["pay_time"])
def downgrade() -> None:
op.drop_table("cps_order")
op.drop_table("cps_activity")
op.drop_table("cps_group")
@@ -0,0 +1,26 @@
"""merge ad_revenue_report_cols and store_mapping_tb_dl_invalid heads
Revision ID: d4e68464761d
Revises: ad_revenue_report_cols, store_mapping_tb_dl_invalid
Create Date: 2026-06-15 21:55:58.115692
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = 'd4e68464761d'
down_revision: Union[str, Sequence[str], None] = ('ad_revenue_report_cols', 'store_mapping_tb_dl_invalid')
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
pass
def downgrade() -> None:
pass
@@ -0,0 +1,32 @@
"""store_mapping 加京东 deeplink 失效标记列 jd_deeplink_invalid_at
Revision ID: store_mapping_jd_dl_invalid
Revises: store_mapping_tb_dl_invalid
Create Date: 2026-06-15 00:00:00.000000
京东同淘宝: 缓存的店内搜索 deeplink 会失效(打开是"当前门店超出配送范围"页)。pricebot 比价撞到
失效页时回退正常搜店, 并 server→server 通知把该 storeId 的 deeplink 标记失效。本列记失效时刻
(NULL=有效); lookup 反查时过滤掉已失效的京东候选, 不再返回坏 deeplink。
与淘宝 taobao_deeplink_invalid_at 对称; 用时间戳而非布尔: 留痕可审计、可统计失效率, 不销毁原 deeplink。
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = 'store_mapping_jd_dl_invalid'
down_revision: Union[str, Sequence[str], None] = 'store_mapping_tb_dl_invalid'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
with op.batch_alter_table('store_mapping', schema=None) as batch_op:
batch_op.add_column(sa.Column('jd_deeplink_invalid_at', sa.DateTime(timezone=True), nullable=True))
def downgrade() -> None:
with op.batch_alter_table('store_mapping', schema=None) as batch_op:
batch_op.drop_column('jd_deeplink_invalid_at')
@@ -0,0 +1,33 @@
"""store_mapping 加淘宝 deeplink 失效标记列 taobao_deeplink_invalid_at
Revision ID: store_mapping_tb_dl_invalid
Revises: coupon_engage_per_package
Create Date: 2026-06-15 00:00:00.000000
缓存的淘宝店内搜索 deeplink 会失效(打开是"页面出错了"降级页)。pricebot 比价撞到错误页时
回退正常搜店, 并 server→server 通知把该 shopId 的 deeplink 标记失效。本列记失效时刻
(NULL=有效); lookup 反查时过滤掉已失效的淘宝候选, 不再返回坏 deeplink。
只加淘宝一列(当前只接淘宝); 用时间戳而非布尔: 留痕可审计、可统计失效率, 且不销毁原 deeplink。
无需索引: 过滤总叠在 name_taobao== 之后, 候选集已小。
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = 'store_mapping_tb_dl_invalid'
down_revision: Union[str, Sequence[str], None] = 'coupon_engage_per_package'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
with op.batch_alter_table('store_mapping', schema=None) as batch_op:
batch_op.add_column(sa.Column('taobao_deeplink_invalid_at', sa.DateTime(timezone=True), nullable=True))
def downgrade() -> None:
with op.batch_alter_table('store_mapping', schema=None) as batch_op:
batch_op.drop_column('taobao_deeplink_invalid_at')
+4
View File
@@ -14,10 +14,12 @@ from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware from fastapi.middleware.cors import CORSMiddleware
from app.admin.routers.ad_audit import router as ad_audit_router from app.admin.routers.ad_audit import router as ad_audit_router
from app.admin.routers.ad_revenue import router as ad_revenue_router
from app.admin.routers.admins import router as admins_router from app.admin.routers.admins import router as admins_router
from app.admin.routers.audit import router as audit_router from app.admin.routers.audit import router as audit_router
from app.admin.routers.auth import router as auth_router from app.admin.routers.auth import router as auth_router
from app.admin.routers.config import router as config_router from app.admin.routers.config import router as config_router
from app.admin.routers.cps import router as cps_router
from app.admin.routers.dashboard import router as dashboard_router from app.admin.routers.dashboard import router as dashboard_router
from app.admin.routers.ops_stat_config import router as ops_stat_config_router from app.admin.routers.ops_stat_config import router as ops_stat_config_router
from app.admin.routers.feedback import router as feedback_router from app.admin.routers.feedback import router as feedback_router
@@ -89,4 +91,6 @@ admin_app.include_router(feedback_router)
admin_app.include_router(admins_router) admin_app.include_router(admins_router)
admin_app.include_router(audit_router) admin_app.include_router(audit_router)
admin_app.include_router(config_router) admin_app.include_router(config_router)
admin_app.include_router(cps_router)
admin_app.include_router(ad_audit_router) admin_app.include_router(ad_audit_router)
admin_app.include_router(ad_revenue_router)
+25 -5
View File
@@ -67,6 +67,8 @@ def _reward_video_rows(
"scene": "reward_video", "scene": "reward_video",
"record_id": rec.id, "record_id": rec.id,
"user_id": rec.user_id, "user_id": rec.user_id,
"app_env": rec.app_env,
"our_code_id": rec.our_code_id,
"created_at": rec.created_at, "created_at": rec.created_at,
"status": rec.status, "status": rec.status,
"ecpm": rec.ecpm_raw, "ecpm": rec.ecpm_raw,
@@ -86,6 +88,8 @@ def _reward_video_rows(
"scene": "reward_video", "scene": "reward_video",
"record_id": rec.id, "record_id": rec.id,
"user_id": rec.user_id, "user_id": rec.user_id,
"app_env": rec.app_env,
"our_code_id": rec.our_code_id,
"created_at": rec.created_at, "created_at": rec.created_at,
"status": rec.status, "status": rec.status,
"ecpm": rec.ecpm_raw, "ecpm": rec.ecpm_raw,
@@ -150,6 +154,8 @@ def _feed_rows(db: Session, *, date: str, user_id: int | None) -> list[dict]:
"scene": "feed", "scene": "feed",
"record_id": rec.id, "record_id": rec.id,
"user_id": rec.user_id, "user_id": rec.user_id,
"app_env": rec.app_env,
"our_code_id": rec.our_code_id,
"created_at": rec.created_at, "created_at": rec.created_at,
"status": rec.status, "status": rec.status,
"ecpm": rec.ecpm_raw, "ecpm": rec.ecpm_raw,
@@ -168,6 +174,8 @@ def _feed_rows(db: Session, *, date: str, user_id: int | None) -> list[dict]:
"scene": "feed", "scene": "feed",
"record_id": rec.id, "record_id": rec.id,
"user_id": rec.user_id, "user_id": rec.user_id,
"app_env": rec.app_env,
"our_code_id": rec.our_code_id,
"created_at": rec.created_at, "created_at": rec.created_at,
"status": rec.status, "status": rec.status,
"ecpm": rec.ecpm_raw, "ecpm": rec.ecpm_raw,
@@ -184,6 +192,22 @@ def _feed_rows(db: Session, *, date: str, user_id: int | None) -> list[dict]:
return rows return rows
def audit_rows(
db: Session, *, date: str, user_id: int | None, scene: str | None = None
) -> list[dict]:
"""当日逐条发奖复算行(未排序)。scene: None=两类 / "reward_video" / "feed"
每行含 `app_env`/`our_code_id`/`expected_coin`/`actual_coin` 等,供金币审计逐条对账,
也供广告收益报表把「应发/实发」按 用户×类型×应用×代码位 聚合(见 ad_revenue,复用同一复算口径)。
"""
rows: list[dict] = []
if scene in (None, "reward_video"):
rows.extend(_reward_video_rows(db, date=date, user_id=user_id))
if scene in (None, "feed"):
rows.extend(_feed_rows(db, date=date, user_id=user_id))
return rows
def ad_coin_audit( def ad_coin_audit(
db: Session, db: Session,
*, *,
@@ -200,11 +224,7 @@ def ad_coin_audit(
影响;`items` 才是展示集(only_mismatch 时只取 ✗ 行)按 created_at 倒序截断到 limit。 影响;`items` 才是展示集(only_mismatch 时只取 ✗ 行)按 created_at 倒序截断到 limit。
份序号在全天数据上已算好,limit 只影响展示条数、不影响 expected 复算正确性。 份序号在全天数据上已算好,limit 只影响展示条数、不影响 expected 复算正确性。
""" """
rows: list[dict] = [] rows = audit_rows(db, date=date, user_id=user_id, scene=scene)
if scene in (None, "reward_video"):
rows.extend(_reward_video_rows(db, date=date, user_id=user_id))
if scene in (None, "feed"):
rows.extend(_feed_rows(db, date=date, user_id=user_id))
rows.sort(key=lambda r: (r["created_at"], r["record_id"]), reverse=True) rows.sort(key=lambda r: (r["created_at"], r["record_id"]), reverse=True)
total = len(rows) total = len(rows)
+239
View File
@@ -0,0 +1,239 @@
"""admin 广告收益报表:按 用户 / 日期 / 广告类型 / 应用 / 代码位 聚合(单表含发奖对账)。
只读。聚合键 = user_id × ad_type × app_env × our_code_id;每组一行同时给出:
- 展示条数 + 收益:`ad_ecpm_record`(每行 = 客户端一次广告展示;收益 = Σ eCPM元 ÷ 1000)。
激励视频每次展示上报一行;信息流轮播每条展示各上报一行(每条独立 id,不复用会话)。
- 应发金币 / 实发金币:复用金币审计的**逐条复算**(`ad_audit.audit_rows`,与正式发奖同一公式口径,
不另写公式),把每条发奖记录的 expected/actual 按同维度求和;`matched` = 组内**逐条**全部一致
(任一条不符该组即不符,不用「应发和==实发和」以免互相抵消掩盖错误)。**不改发奖逻辑**,只读复算。
展示与发奖来自不同表,做并集:有展示无发奖(用户中途关 / 未达发奖)、有发奖无展示
(未上报 eCPM)都各自成行。app_env/our_code_id 旧数据为 NULL → 归到「来源未知」组。
⚠️ 局限:① 历史 Draw 发奖混在 ad_feed_reward_record 无类型标记,金币侧统一记 `feed`(迁移后 Draw
不再产生新数据)。② 聚合级只能看出「某组应发≠实发」,定位到具体哪条仍需逐条审计接口(ad-coin-audit)。
"""
from __future__ import annotations
from datetime import date as _date, datetime, timedelta, timezone
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.admin.repositories import ad_audit
from app.core import rewards
from app.models.ad_ecpm import AdEcpmRecord
def _cn_hour(dt: datetime) -> int:
"""created_at(UTC 口径)→ 北京时间小时(023)。naive 当 UTC 处理(sqlite),tz-aware 直接换算(pg)。"""
if dt.tzinfo is None:
dt = dt.replace(tzinfo=timezone.utc)
return dt.astimezone(rewards.CN_TZ).hour
def _key(
report_date: str,
user_id: int,
ad_type: str,
app_env: str | None,
our_code_id: str | None,
hour: int | None,
) -> tuple:
return (report_date, user_id, ad_type, app_env or None, our_code_id or None, hour)
def _date_range(date_from: str, date_to: str) -> list[str]:
"""闭区间内逐日 'YYYY-MM-DD' 串(含首尾)。date_from > date_to 时返回空。"""
d0 = _date.fromisoformat(date_from)
d1 = _date.fromisoformat(date_to)
out: list[str] = []
d = d0
while d <= d1:
out.append(d.isoformat())
d += timedelta(days=1)
return out
# 审计行的 scene 与报表 ad_type 一一对应
_SCENE_TO_AD_TYPE = {"reward_video": "reward_video", "feed": "feed"}
def ad_revenue_report(
db: Session,
*,
date_from: str,
date_to: str,
user_id: int | None = None,
ad_type: str | None = None,
granularity: str = "day",
limit: int = 500,
) -> dict:
"""日期区间(北京时间,闭区间)广告收益聚合 + 发奖对账。单日时 date_from==date_to。
聚合键含**日期**:report_date × user × ad_type × app_env × our_code_id(× 北京小时,granularity=hour)。
ad_type: None=全部 / reward_video / feed / draw。
granularity: "day"=按天 / "hour"=按小时(聚合键再加北京小时 0–23,每组一行)。
limit 只截断展示明细,total 与 total_* / daily 在全量上统计(不受 limit 影响),数字始终可信。
返回额外含 `daily`(按日期汇总的展示/收益/应发/实发,供前端按天趋势图;不受 limit 影响)。
注:按小时下,展示按 ecpm 记录的小时、金币按发奖记录的小时各自归桶——S2S 回调可能比展示晚
一会儿,故同一次广告的展示与金币偶尔落相邻小时(按天则一致)。
"""
by_hour = granularity == "hour"
groups: dict[tuple, dict] = {}
def _grp(key: tuple) -> dict:
g = groups.get(key)
if g is None:
rdate, uid, atype, app_env, code_id, hour = key
g = {
"report_date": rdate,
"user_id": uid,
"ad_type": atype,
"app_env": app_env,
"our_code_id": code_id,
"hour": hour,
"impressions": 0,
"revenue_yuan": 0.0,
"expected_coin": 0,
"actual_coin": 0,
"adns": set(),
"impression_records": [], # 该组逐条展示明细(展开下钻用)
"records": [], # 该组逐条发奖复算明细(展开下钻用)
}
groups[key] = g
return g
# 1) 展示条数 + 收益 ← ad_ecpm_record(report_date 闭区间;字符串 YYYY-MM-DD 字典序即日期序)
stmt = select(AdEcpmRecord).where(
AdEcpmRecord.report_date >= date_from,
AdEcpmRecord.report_date <= date_to,
)
if user_id is not None:
stmt = stmt.where(AdEcpmRecord.user_id == user_id)
if ad_type is not None:
stmt = stmt.where(AdEcpmRecord.ad_type == ad_type)
for rec in db.execute(stmt).scalars():
hour = _cn_hour(rec.created_at) if by_hour else None
g = _grp(_key(rec.report_date, rec.user_id, rec.ad_type, rec.app_env, rec.our_code_id, hour))
g["impressions"] += 1
# 单次展示收益(元) = eCPM元 ÷ 1000(每千次→单次);用与发奖同源的解析,口径一致。
rev = rewards.parse_ecpm_yuan(rec.ecpm_raw) / 1000.0
g["revenue_yuan"] += rev
if rec.adn:
g["adns"].add(rec.adn)
g["impression_records"].append({
"id": rec.id,
"created_at": rec.created_at,
"ecpm": rec.ecpm_raw,
"revenue_yuan": round(rev, 6),
"adn": rec.adn,
"slot_id": rec.slot_id,
})
# 2) 应发 / 实发金币 ← 复用金币审计逐条复算(同一公式口径),按同维度求和。
# audit_rows 是单日的,区间逐日调用,每天的行归到当天 report_date(语义与单日报表完全一致)。
# ad_type=draw 时审计无对应记录(scene 只有 reward_video/feed),金币侧自然为空。
audit_scene = _SCENE_TO_AD_TYPE.get(ad_type) if ad_type is not None else None
if ad_type is None or audit_scene is not None:
for d in _date_range(date_from, date_to):
for row in ad_audit.audit_rows(db, date=d, user_id=user_id, scene=audit_scene):
atype = _SCENE_TO_AD_TYPE.get(row["scene"], row["scene"])
hour = _cn_hour(row["created_at"]) if by_hour else None
g = _grp(_key(d, row["user_id"], atype, row.get("app_env"), row.get("our_code_id"), hour))
g["expected_coin"] += int(row["expected_coin"])
g["actual_coin"] += int(row["actual_coin"])
# 逐条明细(eCPM/因子1/份数/LT/因子2/应发/实发/一致)——前端展开该组时下钻展示。
g["records"].append({
"record_id": row["record_id"],
"created_at": row["created_at"],
"status": row["status"],
"ecpm": row["ecpm"],
"ecpm_factor": row["ecpm_factor"],
"units": row["units"],
"lt_index_start": row["lt_index_start"],
"lt_index_end": row["lt_index_end"],
"lt_factor_start": row["lt_factor_start"],
"lt_factor_end": row["lt_factor_end"],
"expected_coin": row["expected_coin"],
"actual_coin": row["actual_coin"],
"matched": row["matched"],
})
rows = list(groups.values())
rows.sort(
key=lambda r: (
r["report_date"],
r["user_id"],
r["hour"] if r["hour"] is not None else -1,
r["ad_type"] or "",
r["our_code_id"] or "",
)
)
total_impressions = sum(r["impressions"] for r in rows)
total_expected_coin = sum(r["expected_coin"] for r in rows)
total_actual_coin = sum(r["actual_coin"] for r in rows)
total_revenue_yuan = round(sum(r["revenue_yuan"] for r in rows), 6)
# 按日期汇总(全量,不受 limit):供前端按天趋势图。
daily_map: dict[str, dict] = {}
for r in rows:
d = daily_map.get(r["report_date"])
if d is None:
d = {
"date": r["report_date"],
"impressions": 0,
"revenue_yuan": 0.0,
"expected_coin": 0,
"actual_coin": 0,
}
daily_map[r["report_date"]] = d
d["impressions"] += r["impressions"]
d["revenue_yuan"] += r["revenue_yuan"]
d["expected_coin"] += r["expected_coin"]
d["actual_coin"] += r["actual_coin"]
daily = [
{**d, "revenue_yuan": round(d["revenue_yuan"], 6)}
for d in sorted(daily_map.values(), key=lambda x: x["date"])
]
items = [
{
"report_date": r["report_date"],
"user_id": r["user_id"],
"ad_type": r["ad_type"],
"app_env": r["app_env"],
"our_code_id": r["our_code_id"],
"hour": r["hour"],
"impressions": r["impressions"],
"revenue_yuan": round(r["revenue_yuan"], 6),
"expected_coin": r["expected_coin"],
"actual_coin": r["actual_coin"],
# 组内**逐条**全部一致才记一致——不能用「应发和==实发和」,否则一条多发+一条少发会互相
# 抵消、求和相等被误判为 ✓,掩盖真实发奖错误。纯展示无发奖记录的组 all([]) → True。
"matched": all(rec["matched"] for rec in r["records"]),
"adns": sorted(r["adns"]),
"impression_records": sorted(
r["impression_records"], key=lambda x: (x["created_at"], x["id"])
),
"records": sorted(r["records"], key=lambda x: (x["created_at"], x["record_id"])),
}
for r in rows[:limit]
]
return {
"total": len(rows),
"truncated": len(rows) > limit,
"total_impressions": total_impressions,
"total_revenue_yuan": total_revenue_yuan,
"total_expected_coin": total_expected_coin,
"total_actual_coin": total_actual_coin,
"mismatch_count": sum(
1 for r in rows if not all(rec["matched"] for rec in r["records"])
),
"daily": daily,
"items": items,
}
+287
View File
@@ -0,0 +1,287 @@
"""admin CPS 数据访问:群/活动 CRUD + 美团订单拉单入库(对账) + 按群统计聚合。
美团 query_order 返回的金额是「元」字符串、时间是秒级时间戳,这里统一转「分」+ tz-aware。
统计在 Python 侧聚合(订单量级小、admin 低频),逻辑清晰、跨 PG/SQLite 无 SQL 方言坑。
"""
from __future__ import annotations
from datetime import datetime, timezone
from decimal import Decimal, InvalidOperation
from uuid import uuid4
from sqlalchemy import desc, select
from sqlalchemy.orm import Session
from app.admin.repositories.queries import _as_utc, offset_paginate
from app.integrations import meituan
from app.repositories import cps_link as cps_link_repo
from app.models.cps_activity import CpsActivity
from app.models.cps_group import CpsGroup
from app.models.cps_order import CpsOrder
# 美团订单状态:取消(4)/风控(5)不计佣金;结算(6)为佣金真正到账
_INVALID_STATUS = {"4", "5"}
_SETTLED_STATUS = "6"
# ───────────── 单位换算 ─────────────
def _yuan_to_cents(v: object) -> int | None:
"""「元」字符串/数 → 分。None / 空 / 字面 "null" → None。"""
if v is None:
return None
s = str(v).strip()
if not s or s.lower() == "null":
return None
try:
return int((Decimal(s) * 100).to_integral_value())
except (InvalidOperation, ValueError):
return None
def _ts_to_dt(ts: object) -> datetime | None:
"""秒级时间戳 → tz-aware UTC datetime(绝对时刻,前端按北京展示)。"""
if not ts:
return None
try:
return datetime.fromtimestamp(int(ts), tz=timezone.utc)
except (ValueError, OSError, TypeError):
return None
# ───────────── 群 ─────────────
def get_group(db: Session, group_id: int) -> CpsGroup | None:
return db.get(CpsGroup, group_id)
def get_group_by_sid(db: Session, sid: str) -> CpsGroup | None:
return db.execute(select(CpsGroup).where(CpsGroup.sid == sid)).scalar_one_or_none()
def list_groups(
db: Session, *, keyword: str | None = None, status: str | None = None,
limit: int = 20, cursor: int | None = None,
) -> tuple[list[CpsGroup], int | None, int]:
stmt = select(CpsGroup)
if keyword and keyword.strip():
kw = f"%{keyword.strip()}%"
stmt = stmt.where(CpsGroup.name.ilike(kw) | CpsGroup.sid.ilike(kw))
if status:
stmt = stmt.where(CpsGroup.status == status)
return offset_paginate(db, stmt, (desc(CpsGroup.id),), limit=limit, cursor=cursor)
def create_group(
db: Session, *, name: str, sid: str | None = None,
member_count: int | None = None, remark: str | None = None, commit: bool = True,
) -> CpsGroup:
"""建群。sid 留空 → 自动 g<id>(先 flush 拿 id 再回填)。"""
group = CpsGroup(
name=name,
sid=sid or f"tmp{uuid4().hex[:20]}", # 临时唯一占位,留空时下面回填 g<id>
member_count=member_count,
remark=remark,
)
db.add(group)
db.flush()
if not sid:
group.sid = f"g{group.id}"
db.flush()
if commit:
db.commit()
db.refresh(group)
return group
def update_group(
db: Session, group: CpsGroup, *, name: str | None = None,
member_count: int | None = None, status: str | None = None,
remark: str | None = None, commit: bool = True,
) -> CpsGroup:
if name is not None:
group.name = name
if member_count is not None:
group.member_count = member_count
if status is not None:
group.status = status
if remark is not None:
group.remark = remark
if commit:
db.commit()
db.refresh(group)
else:
db.flush()
return group
# ───────────── 活动 ─────────────
def get_activity(db: Session, activity_id: int) -> CpsActivity | None:
return db.get(CpsActivity, activity_id)
def list_activities(
db: Session, *, platform: str | None = None, status: str | None = None,
limit: int = 20, cursor: int | None = None,
) -> tuple[list[CpsActivity], int | None, int]:
stmt = select(CpsActivity)
if platform:
stmt = stmt.where(CpsActivity.platform == platform)
if status:
stmt = stmt.where(CpsActivity.status == status)
return offset_paginate(db, stmt, (desc(CpsActivity.id),), limit=limit, cursor=cursor)
def create_activity(
db: Session, *, name: str, platform: str = "meituan", act_id: str | None = None,
product_view_sign: str | None = None, remark: str | None = None, commit: bool = True,
) -> CpsActivity:
activity = CpsActivity(
name=name, platform=platform, act_id=act_id,
product_view_sign=product_view_sign, remark=remark,
)
db.add(activity)
if commit:
db.commit()
db.refresh(activity)
else:
db.flush()
return activity
# ───────────── 订单(对账) ─────────────
def _map_order_fields(r: dict) -> dict:
"""美团 query_order 单条 dataList → CpsOrder 字段(金额转分、时间转 datetime)。"""
pn = r.get("productName")
if pn and len(pn) > 500:
pn = pn[:500]
return {
"sid": (r.get("sid") or None),
"act_id": str(r["actId"]) if r.get("actId") is not None else None,
"biz_line": r.get("businessLine"),
"trade_type": r.get("tradeType"),
"pay_price_cents": _yuan_to_cents(r.get("payPrice")),
"commission_cents": _yuan_to_cents(r.get("profit")),
"commission_rate": str(r["commissionRate"]) if r.get("commissionRate") is not None else None,
"refund_price_cents": _yuan_to_cents(r.get("refundPrice")),
"refund_profit_cents": _yuan_to_cents(r.get("refundProfit")),
"mt_status": str(r["status"]) if r.get("status") is not None else None,
"invalid_reason": (r.get("invalidReason") or None),
"product_name": pn or None,
"pay_time": _ts_to_dt(r.get("payTime")),
"mt_update_time": _ts_to_dt(r.get("updateTime")),
"raw": r,
}
def reconcile_orders(
db: Session, *, start_time: int, end_time: int,
query_time_type: int = 1, sid: str | None = None, max_pages: int = 200,
) -> dict:
"""调美团 query_order 分页拉单 → 按 order_id upsert。返回 {fetched, inserted, updated, pages}。
订单状态会随时间变(付款→完成→结算/退款),重复拉同一单则更新。max_pages 防异常死循环。
"""
fetched = inserted = updated = pages = 0
page = 1
while page <= max_pages:
resp = meituan.query_order(
sid=sid, start_time=start_time, end_time=end_time,
query_time_type=query_time_type, page=page, limit=100,
)
rows = ((resp.get("data") or {}).get("dataList")) or []
if not rows:
break
pages += 1
for r in rows:
order_id = str(r.get("orderId") or "").strip()
if not order_id:
continue
fetched += 1
fields = _map_order_fields(r)
existing = db.execute(
select(CpsOrder).where(CpsOrder.order_id == order_id)
).scalar_one_or_none()
if existing is None:
db.add(CpsOrder(order_id=order_id, **fields))
inserted += 1
else:
for k, v in fields.items():
setattr(existing, k, v)
updated += 1
if len(rows) < 100:
break
page += 1
db.commit()
return {"fetched": fetched, "inserted": inserted, "updated": updated, "pages": pages}
def list_orders(
db: Session, *, sid: str | None = None, mt_status: str | None = None,
limit: int = 20, cursor: int | None = None,
) -> tuple[list[CpsOrder], int | None, int]:
stmt = select(CpsOrder)
if sid:
stmt = stmt.where(CpsOrder.sid == sid)
if mt_status:
stmt = stmt.where(CpsOrder.mt_status == mt_status)
return offset_paginate(db, stmt, (desc(CpsOrder.id),), limit=limit, cursor=cursor)
# ───────────── 统计(按群聚合) ─────────────
def group_stats(
db: Session, *, date_from: datetime | None = None, date_to: datetime | None = None,
) -> list[dict]:
"""按 sid 聚合订单 + join 群信息。已建但本期无单的活跃群也列出(全 0)。
未归群的 sid(历史/其它来源)单独成行 group_id=None。按预估佣金降序。
"""
stmt = select(CpsOrder)
if date_from is not None:
stmt = stmt.where(CpsOrder.pay_time >= _as_utc(date_from))
if date_to is not None:
stmt = stmt.where(CpsOrder.pay_time <= _as_utc(date_to))
orders = list(db.execute(stmt).scalars().all())
groups = {g.sid: g for g in db.execute(select(CpsGroup)).scalars().all()}
buckets: dict[str | None, list[CpsOrder]] = {}
for o in orders:
buckets.setdefault(o.sid, []).append(o)
rows: list[dict] = []
for sid, items in buckets.items():
g = groups.get(sid) if sid else None
valid = [o for o in items if o.mt_status not in _INVALID_STATUS]
settled = [o for o in items if o.mt_status == _SETTLED_STATUS]
canceled = [o for o in items if o.mt_status in _INVALID_STATUS]
rows.append({
"group_id": g.id if g else None,
"sid": sid,
"name": g.name if g else (sid or "(无 sid 归属)"),
"member_count": g.member_count if g else None,
"order_count": len(valid),
"settled_count": len(settled),
"canceled_count": len(canceled),
"gmv_cents": sum(o.pay_price_cents or 0 for o in valid),
"est_commission_cents": sum(o.commission_cents or 0 for o in valid),
"settled_commission_cents": sum(o.commission_cents or 0 for o in settled),
})
# 已建的活跃群但本期无单 → 补 0 行,让运营看到全部群
seen = set(buckets.keys())
for sid, g in groups.items():
if sid not in seen and g.status == "active":
rows.append({
"group_id": g.id, "sid": sid, "name": g.name,
"member_count": g.member_count, "order_count": 0,
"settled_count": 0, "canceled_count": 0, "gmv_cents": 0,
"est_commission_cents": 0, "settled_commission_cents": 0,
})
# 合并点击数据(按 group_id):点击只对生成过我们短链的群有,未归群行(group_id=None)恒 0
clicks = cps_link_repo.click_stats_by_group(db, date_from=date_from, date_to=date_to)
for r in rows:
c = clicks.get(r["group_id"]) if r["group_id"] is not None else None
r["click_pv"] = c["pv"] if c else 0
r["click_uv"] = c["uv"] if c else 0
rows.sort(key=lambda x: x["est_commission_cents"], reverse=True)
return rows
+4 -10
View File
@@ -286,10 +286,11 @@ def list_feedbacks(
sort_order: str = "desc", sort_order: str = "desc",
limit: int = 20, limit: int = 20,
cursor: int | None = None, cursor: int | None = None,
) -> tuple[list[Feedback], int | None]: ) -> tuple[list[Feedback], int | None, int]:
"""反馈工单列表。支持 状态 / 用户ID / 内容模糊 / 提交时间范围 筛选,按 id·提交时间排序。 """反馈工单列表。支持 状态 / 用户ID / 内容模糊 / 提交时间范围 筛选,按 id·提交时间排序。
**offset 分页**(cursor=offset):任意列排序下游标语义统一(同 [list_users]),代价是翻页期间 **offset 分页**(cursor=offset):任意列排序下游标语义统一(同 [list_users]),代价是翻页期间
数据变动可能错位一条——admin 低频场景可接受。created_at 为 timestamptz,日期入参统一转 tz-aware UTC 比较。""" 数据变动可能错位一条——admin 低频场景可接受。返回 (items, next_cursor, total),total 供页码分页。
created_at 为 timestamptz,日期入参统一转 tz-aware UTC 比较。"""
stmt = select(Feedback) stmt = select(Feedback)
if status: if status:
stmt = stmt.where(Feedback.status == status) stmt = stmt.where(Feedback.status == status)
@@ -306,14 +307,7 @@ def list_feedbacks(
sort_col = sort_cols.get(sort_by, Feedback.id) sort_col = sort_cols.get(sort_by, Feedback.id)
order_fn = asc if sort_order == "asc" else desc order_fn = asc if sort_order == "asc" else desc
id_order = asc(Feedback.id) if sort_order == "asc" else desc(Feedback.id) id_order = asc(Feedback.id) if sort_order == "asc" else desc(Feedback.id)
stmt = stmt.order_by(order_fn(sort_col), id_order) return offset_paginate(db, stmt, (order_fn(sort_col), id_order), limit=limit, cursor=cursor)
offset = max(cursor or 0, 0)
rows = list(db.execute(stmt.offset(offset).limit(limit + 1)).scalars().all())
has_more = len(rows) > limit
items = rows[:limit]
next_cursor = offset + limit if has_more else None
return items, next_cursor
def get_withdraw_by_out_bill_no(db: Session, out_bill_no: str) -> WithdrawOrder | None: def get_withdraw_by_out_bill_no(db: Session, out_bill_no: str) -> WithdrawOrder | None:
+75
View File
@@ -0,0 +1,75 @@
"""admin 广告收益报表:按 用户/日期/广告类型/应用/代码位 聚合 展示条数 / 收益 / 金币。
任意已登录 admin 可看(只读,不涉及资金操作)。聚合逻辑在 app/admin/repositories/ad_revenue.py。
"""
from __future__ import annotations
from datetime import date as _date
from typing import Annotated
from fastapi import APIRouter, Depends, HTTPException, Query
from app.admin.deps import AdminDb, get_current_admin
from app.admin.repositories import ad_revenue
from app.admin.schemas.ad_revenue import AdRevenueDaily, AdRevenueReportOut, AdRevenueRow
from app.core.rewards import cn_today
router = APIRouter(
prefix="/admin/api/ad-revenue-report",
tags=["admin-ad-revenue-report"],
dependencies=[Depends(get_current_admin)],
)
# 区间最大跨度(天);超出拒绝,避免审计页一次拉过多天(逐日审计 + 大查询)拖垮接口。
_MAX_RANGE_DAYS = 92
def _parse_day(value: str | None, *, field: str, default: _date) -> _date:
if value is None:
return default
try:
return _date.fromisoformat(value)
except ValueError as e:
raise HTTPException(status_code=422, detail=f"{field} 需为 YYYY-MM-DD") from e
@router.get("", response_model=AdRevenueReportOut, summary="广告收益报表(按 日期区间/用户/类型/应用/代码位 聚合)")
def get_ad_revenue_report(
db: AdminDb,
date_from: Annotated[str | None, Query(description="起始日 北京时间 YYYY-MM-DD,默认今天")] = None,
date_to: Annotated[str | None, Query(description="结束日 北京时间 YYYY-MM-DD,闭区间,默认=date_from")] = None,
user_id: Annotated[int | None, Query(description="只看某用户;不传=全部用户")] = None,
ad_type: Annotated[
str | None,
Query(description="reward_video / feed / draw;不传=全部类型"),
] = None,
granularity: Annotated[
str, Query(description="day=按天 / hour=按小时(北京时间);区间>1 天建议用 day")
] = "day",
limit: Annotated[int, Query(ge=1, le=1000)] = 500,
) -> AdRevenueReportOut:
today = cn_today()
d_from = _parse_day(date_from, field="date_from", default=today)
d_to = _parse_day(date_to, field="date_to", default=d_from)
if d_to < d_from:
raise HTTPException(status_code=422, detail="date_to 不能早于 date_from")
if (d_to - d_from).days + 1 > _MAX_RANGE_DAYS:
raise HTTPException(status_code=422, detail=f"区间最长 {_MAX_RANGE_DAYS}")
result = ad_revenue.ad_revenue_report(
db, date_from=d_from.isoformat(), date_to=d_to.isoformat(),
user_id=user_id, ad_type=ad_type, granularity=granularity, limit=limit,
)
return AdRevenueReportOut(
date_from=d_from.isoformat(),
date_to=d_to.isoformat(),
daily=[AdRevenueDaily(**d) for d in result["daily"]],
total=result["total"],
truncated=result["truncated"],
total_impressions=result["total_impressions"],
total_revenue_yuan=result["total_revenue_yuan"],
total_expected_coin=result["total_expected_coin"],
total_actual_coin=result["total_actual_coin"],
mismatch_count=result["mismatch_count"],
items=[AdRevenueRow(**r) for r in result["items"]],
)
+3
View File
@@ -43,6 +43,9 @@ def _validate(key: str, value: Any) -> None:
for k, v in value.items() for k, v in value.items()
): ):
raise ValueError("需为 {字符串: 整数} 映射") raise ValueError("需为 {字符串: 整数} 映射")
elif t == "bool":
if not isinstance(value, bool):
raise ValueError("需为布尔值")
def _item(db, key: str) -> ConfigItemOut: def _item(db, key: str) -> ConfigItemOut:
+268
View File
@@ -0,0 +1,268 @@
"""admin CPS 分发与对账:群(sid)管理 + 活动池 + 生成带 sid 券链接 + 美团订单对账 + 统计。
当前仅接美团(get_referral_link / query_order)。淘宝/京东待各自联盟凭证。
群/活动管理 = operator;订单对账(涉佣金) = finance;只读列表/统计 = 登录即可。
"""
from __future__ import annotations
import time
from datetime import datetime, timedelta, timezone
from typing import Annotated
from fastapi import APIRouter, Depends, HTTPException, Query, Request
from app.admin.audit import write_audit
from app.admin.deps import AdminDb, get_client_ip, get_current_admin, require_role
from app.admin.repositories import cps as cps_repo
from app.admin.schemas.common import CursorPage
from app.core.config import settings
from app.repositories import cps_link as cps_link_repo
from app.admin.schemas.cps import (
CpsActivityCreate,
CpsActivityOut,
CpsGroupCreate,
CpsGroupOut,
CpsGroupStat,
CpsGroupUpdate,
CpsOrderOut,
CpsReconcileResult,
CpsReferralLinkOut,
CpsReferralLinkRequest,
CpsStatsOut,
)
from app.integrations import meituan
from app.integrations.meituan import MeituanCpsError
from app.models.admin import AdminUser
router = APIRouter(
prefix="/admin/api/cps",
tags=["admin-cps"],
dependencies=[Depends(get_current_admin)],
)
# ───────────── 群(sid) ─────────────
@router.get("/groups", response_model=CursorPage[CpsGroupOut], summary="群(sid)列表")
def list_groups(
db: AdminDb,
keyword: Annotated[str | None, Query(max_length=100)] = None,
status: Annotated[str | None, Query(pattern="^(active|archived)$")] = None,
limit: Annotated[int, Query(ge=1, le=100)] = 20,
cursor: Annotated[int | None, Query()] = None,
) -> CursorPage[CpsGroupOut]:
items, next_cursor, total = cps_repo.list_groups(
db, keyword=keyword, status=status, limit=limit, cursor=cursor
)
return CursorPage(
items=[CpsGroupOut.model_validate(g) for g in items],
next_cursor=next_cursor,
total=total,
)
@router.post("/groups", response_model=CpsGroupOut, summary="新建群(分配 sid)")
def create_group(
body: CpsGroupCreate,
request: Request,
admin: Annotated[AdminUser, Depends(require_role("operator"))],
db: AdminDb,
) -> CpsGroupOut:
if body.sid and cps_repo.get_group_by_sid(db, body.sid) is not None:
raise HTTPException(status_code=409, detail="sid 已存在")
group = cps_repo.create_group(
db, name=body.name, sid=body.sid, member_count=body.member_count,
remark=body.remark, commit=False,
)
write_audit(
db, admin, action="cps.group.create", target_type="cps_group", target_id=group.id,
detail={"sid": group.sid, "name": group.name}, ip=get_client_ip(request), commit=False,
)
db.commit()
db.refresh(group)
return CpsGroupOut.model_validate(group)
@router.patch("/groups/{group_id}", response_model=CpsGroupOut, summary="编辑群")
def update_group(
group_id: int,
body: CpsGroupUpdate,
request: Request,
admin: Annotated[AdminUser, Depends(require_role("operator"))],
db: AdminDb,
) -> CpsGroupOut:
group = cps_repo.get_group(db, group_id)
if group is None:
raise HTTPException(status_code=404, detail="群不存在")
cps_repo.update_group(
db, group, name=body.name, member_count=body.member_count,
status=body.status, remark=body.remark, commit=False,
)
write_audit(
db, admin, action="cps.group.update", target_type="cps_group", target_id=group_id,
detail=body.model_dump(exclude_none=True), ip=get_client_ip(request), commit=False,
)
db.commit()
db.refresh(group)
return CpsGroupOut.model_validate(group)
# ───────────── 活动 ─────────────
@router.get("/activities", response_model=CursorPage[CpsActivityOut], summary="活动列表")
def list_activities(
db: AdminDb,
platform: Annotated[str | None, Query(max_length=20)] = None,
status: Annotated[str | None, Query(pattern="^(active|archived)$")] = None,
limit: Annotated[int, Query(ge=1, le=100)] = 20,
cursor: Annotated[int | None, Query()] = None,
) -> CursorPage[CpsActivityOut]:
items, next_cursor, total = cps_repo.list_activities(
db, platform=platform, status=status, limit=limit, cursor=cursor
)
return CursorPage(
items=[CpsActivityOut.model_validate(a) for a in items],
next_cursor=next_cursor,
total=total,
)
@router.post("/activities", response_model=CpsActivityOut, summary="新建活动")
def create_activity(
body: CpsActivityCreate,
request: Request,
admin: Annotated[AdminUser, Depends(require_role("operator"))],
db: AdminDb,
) -> CpsActivityOut:
if not body.act_id and not body.product_view_sign:
raise HTTPException(status_code=400, detail="act_id 或 product_view_sign 必填其一")
activity = cps_repo.create_activity(
db, name=body.name, platform=body.platform, act_id=body.act_id,
product_view_sign=body.product_view_sign, remark=body.remark, commit=False,
)
write_audit(
db, admin, action="cps.activity.create", target_type="cps_activity", target_id=activity.id,
detail={"name": activity.name, "act_id": activity.act_id, "platform": activity.platform},
ip=get_client_ip(request), commit=False,
)
db.commit()
db.refresh(activity)
return CpsActivityOut.model_validate(activity)
# ───────────── 转链 ─────────────
@router.post("/referral-link", response_model=CpsReferralLinkOut, summary="生成群发短链(带 sid + 点击统计)")
def generate_referral_link(
body: CpsReferralLinkRequest,
request: Request,
admin: Annotated[AdminUser, Depends(require_role("operator"))],
db: AdminDb,
) -> CpsReferralLinkOut:
group = cps_repo.get_group(db, body.group_id)
if group is None:
raise HTTPException(status_code=404, detail="群不存在")
activity = cps_repo.get_activity(db, body.activity_id)
if activity is None:
raise HTTPException(status_code=404, detail="活动不存在")
if activity.platform != "meituan":
raise HTTPException(status_code=400, detail=f"当前仅支持美团转链(活动平台={activity.platform})")
try:
# 全要:短链作跳转目标(微信可打开),长链/deeplink 作参考/备用
resp = meituan.get_referral_link(
act_id=activity.act_id,
product_view_sign=activity.product_view_sign,
sid=group.sid,
link_type_list=[1, 2, 3],
)
except MeituanCpsError as e:
raise HTTPException(status_code=502, detail=f"美团转链失败: {e}") from e
link_map = {str(k): v for k, v in (resp.get("referralLinkMap") or {}).items()}
if not link_map and resp.get("data"):
link_map = {"1": resp["data"]}
# 跳转目标优先短链(2,微信可打开)→ 长链(1)→ deeplink(3)
target_url = link_map.get("2") or link_map.get("1") or link_map.get("3")
if not target_url:
raise HTTPException(status_code=502, detail="美团未返回有效链接(检查 actId 是否在推广有效期)")
# 存我们的短链:发群用 /c/{code},用户点 → 我们记点击 → 302 跳 target_url
link = cps_link_repo.create_link(
db, group_id=group.id, activity_id=activity.id, sid=group.sid,
target_url=target_url, platform=activity.platform, commit=False,
)
write_audit(
db, admin, action="cps.referral_link.generate", target_type="cps_link", target_id=link.id,
detail={"sid": group.sid, "activity_id": activity.id, "code": link.code},
ip=get_client_ip(request), commit=False,
)
db.commit()
db.refresh(link)
base = settings.CPS_REDIRECT_BASE.rstrip("/")
redirect_url = f"{base}/c/{link.code}" if base else f"/c/{link.code}"
return CpsReferralLinkOut(
redirect_url=redirect_url,
code=link.code,
sid=group.sid,
group_name=group.name,
activity_name=activity.name,
target_url=target_url,
link_map=link_map,
)
# ───────────── 订单对账 ─────────────
@router.post("/orders/reconcile", response_model=CpsReconcileResult, summary="拉取美团订单对账")
def reconcile_orders(
request: Request,
admin: Annotated[AdminUser, Depends(require_role("finance"))],
db: AdminDb,
days: Annotated[int, Query(ge=1, le=90)] = 7,
sid: Annotated[str | None, Query(max_length=64)] = None,
) -> CpsReconcileResult:
now = int(time.time())
try:
result = cps_repo.reconcile_orders(
db, start_time=now - days * 86400, end_time=now, sid=sid,
)
except MeituanCpsError as e:
raise HTTPException(status_code=502, detail=f"美团拉单失败: {e}") from e
write_audit(
db, admin, action="cps.order.reconcile", target_type="cps_order", target_id=None,
detail={"days": days, "sid": sid, **result}, ip=get_client_ip(request), commit=True,
)
return CpsReconcileResult(**result)
@router.get("/orders", response_model=CursorPage[CpsOrderOut], summary="订单明细")
def list_orders(
db: AdminDb,
sid: Annotated[str | None, Query(max_length=64)] = None,
mt_status: Annotated[str | None, Query(max_length=8)] = None,
limit: Annotated[int, Query(ge=1, le=100)] = 20,
cursor: Annotated[int | None, Query()] = None,
) -> CursorPage[CpsOrderOut]:
items, next_cursor, total = cps_repo.list_orders(
db, sid=sid, mt_status=mt_status, limit=limit, cursor=cursor
)
return CursorPage(
items=[CpsOrderOut.model_validate(o) for o in items],
next_cursor=next_cursor,
total=total,
)
# ───────────── 统计 ─────────────
@router.get("/stats", response_model=CpsStatsOut, summary="按群对账统计")
def get_stats(
db: AdminDb,
days: Annotated[int, Query(ge=1, le=90)] = 30,
) -> CpsStatsOut:
date_to = datetime.now(timezone.utc)
date_from = date_to - timedelta(days=days)
rows = cps_repo.group_stats(db, date_from=date_from, date_to=date_to)
stats = [CpsGroupStat(**r) for r in rows]
return CpsStatsOut(
groups=stats,
total_order_count=sum(s.order_count for s in stats),
total_est_commission_cents=sum(s.est_commission_cents for s in stats),
total_settled_commission_cents=sum(s.settled_commission_cents for s in stats),
)
+4 -2
View File
@@ -34,7 +34,7 @@ def list_feedbacks(
limit: Annotated[int, Query(ge=1, le=100)] = 20, limit: Annotated[int, Query(ge=1, le=100)] = 20,
cursor: Annotated[int | None, Query()] = None, cursor: Annotated[int | None, Query()] = None,
) -> CursorPage[FeedbackOut]: ) -> CursorPage[FeedbackOut]:
items, next_cursor = queries.list_feedbacks( items, next_cursor, total = queries.list_feedbacks(
db, db,
status=status, status=status,
user_id=user_id, user_id=user_id,
@@ -47,7 +47,9 @@ def list_feedbacks(
cursor=cursor, cursor=cursor,
) )
return CursorPage( return CursorPage(
items=[FeedbackOut.model_validate(f) for f in items], next_cursor=next_cursor, items=[FeedbackOut.model_validate(f) for f in items],
next_cursor=next_cursor,
total=total,
) )
+12 -4
View File
@@ -34,6 +34,7 @@ from app.admin.schemas.wallet import (
from app.core.config import settings from app.core.config import settings
from app.integrations import wxpay from app.integrations import wxpay
from app.models.admin import AdminUser from app.models.admin import AdminUser
from app.repositories import app_config
from app.repositories import wallet as wallet_repo from app.repositories import wallet as wallet_repo
router = APIRouter( router = APIRouter(
@@ -95,7 +96,7 @@ def withdraws_summary(db: AdminDb) -> WithdrawSummaryOut:
summary="提现配置健康检查", summary="提现配置健康检查",
dependencies=[Depends(require_role("finance"))], # 暴露密钥路径/配置,限财务+super dependencies=[Depends(require_role("finance"))], # 暴露密钥路径/配置,限财务+super
) )
def withdraw_health_check() -> WxpayHealthCheckOut: def withdraw_health_check(db: AdminDb) -> WxpayHealthCheckOut:
private_path = wxpay._resolve_config_path(settings.WXPAY_MCH_PRIVATE_KEY_PATH) # noqa: SLF001 private_path = wxpay._resolve_config_path(settings.WXPAY_MCH_PRIVATE_KEY_PATH) # noqa: SLF001
public_path = wxpay._resolve_config_path(settings.WXPAY_PUBLIC_KEY_PATH) # noqa: SLF001 public_path = wxpay._resolve_config_path(settings.WXPAY_PUBLIC_KEY_PATH) # noqa: SLF001
issues: list[str] = [] issues: list[str] = []
@@ -117,8 +118,15 @@ def withdraw_health_check() -> WxpayHealthCheckOut:
issues.append("微信支付基础配置不完整") issues.append("微信支付基础配置不完整")
if not settings.WXPAY_AUTH_NOTIFY_URL: if not settings.WXPAY_AUTH_NOTIFY_URL:
issues.append("免确认授权回调地址未配置") issues.append("免确认授权回调地址未配置")
if not settings.WITHDRAW_AUTO_RECONCILE_ENABLED:
issues.append("自动对账未开启") # 实际是否自动对账 = env 部署总闸(worker 起没起)AND 运营后台 DB 开关(本轮跑不跑)。
worker_running = settings.WITHDRAW_AUTO_RECONCILE_ENABLED
daily_on = bool(app_config.get_value(db, "withdraw_auto_reconcile_enabled"))
auto_reconcile_enabled = worker_running and daily_on
if not worker_running:
issues.append("自动对账 worker 未启动(部署侧 env WITHDRAW_AUTO_RECONCILE_ENABLED=false)")
elif not daily_on:
issues.append("自动对账运营开关已关闭(系统配置页可开)")
return WxpayHealthCheckOut( return WxpayHealthCheckOut(
ok=not issues, ok=not issues,
@@ -131,7 +139,7 @@ def withdraw_health_check() -> WxpayHealthCheckOut:
public_key_exists=public_path.exists(), public_key_exists=public_path.exists(),
public_key_loadable=public_loadable, public_key_loadable=public_loadable,
auth_notify_url_configured=bool(settings.WXPAY_AUTH_NOTIFY_URL), auth_notify_url_configured=bool(settings.WXPAY_AUTH_NOTIFY_URL),
auto_reconcile_enabled=settings.WITHDRAW_AUTO_RECONCILE_ENABLED, auto_reconcile_enabled=auto_reconcile_enabled,
auto_reconcile_interval_sec=settings.WITHDRAW_AUTO_RECONCILE_INTERVAL_SEC, auto_reconcile_interval_sec=settings.WITHDRAW_AUTO_RECONCILE_INTERVAL_SEC,
auto_reconcile_older_than_minutes=settings.WITHDRAW_AUTO_RECONCILE_OLDER_THAN_MINUTES, auto_reconcile_older_than_minutes=settings.WITHDRAW_AUTO_RECONCILE_OLDER_THAN_MINUTES,
issues=issues, issues=issues,
+90
View File
@@ -0,0 +1,90 @@
"""广告收益报表 schemas。
按 用户 / 日期 / 广告类型 / 应用 / 代码位 聚合的只读报表:展示条数、收益(元)、金币、来源。
字段 snake_case;收益按元(float),金币按整数。
"""
from __future__ import annotations
from datetime import datetime
from pydantic import BaseModel, Field
class AdRevenueImpression(BaseModel):
"""聚合行下钻的单条**展示**明细(每次广告展示一条,展开该组时展示)。"""
id: int = Field(..., description="ad_ecpm_record 主键")
created_at: datetime
ecpm: str = Field(..., description="本次展示 eCPM 原始值(分/千次展示)")
revenue_yuan: float = Field(..., description="本次展示预估收益(元)= eCPM元 ÷ 1000")
adn: str | None = Field(None, description="实际填充 ADN 子渠道(pangle/gdt…)")
slot_id: str | None = Field(None, description="底层 mediation rit(非我们配置的广告位 ID)")
class AdRevenueRecord(BaseModel):
"""聚合行下钻的单条发奖复算明细(与金币审计同源,展开该组时展示)。"""
record_id: int
created_at: datetime
status: str = Field(..., description="granted / capped / ecpm_missing")
ecpm: str | None = Field(None, description="本次采用的 eCPM 原始值(分/千次展示)")
ecpm_factor: float | None = Field(None, description="因子1(eCPM 档);非 granted 为空")
units: int = Field(..., description="折算份数:激励视频恒 1;信息流 = 满 10 秒份数")
lt_index_start: int | None = Field(None, description="本条占用「账号累计第几份」的起")
lt_index_end: int | None = Field(None, description="本条占用「账号累计第几份」的止;激励视频 = 起")
lt_factor_start: float | None = Field(None, description="因子2(LT)起值")
lt_factor_end: float | None = Field(None, description="因子2(LT)止值;激励视频 = 起")
expected_coin: int = Field(..., description="按公式复算应发金币")
actual_coin: int = Field(..., description="实际入账金币")
matched: bool = Field(..., description="复算与实发是否一致")
class AdRevenueDaily(BaseModel):
"""按日期汇总的一天(供前端按天趋势图;全量,不受 limit 影响)。"""
date: str = Field(..., description="北京时间 YYYY-MM-DD")
impressions: int = Field(..., description="当天展示条数合计")
revenue_yuan: float = Field(..., description="当天预估收益合计(元)")
expected_coin: int = Field(..., description="当天应发金币合计")
actual_coin: int = Field(..., description="当天实发金币合计")
class AdRevenueRow(BaseModel):
"""一个聚合组(report_date × user × ad_type × app_env × our_code_id)的汇总。"""
report_date: str = Field(..., description="该组所属日期(北京时间 YYYY-MM-DD)")
user_id: int
ad_type: str = Field(..., description="reward_video(激励视频) / feed(信息流) / draw(历史 Draw 信息流)")
app_env: str | None = Field(None, description="我们的应用:prod(傻瓜比价正式) / test(测试应用);旧数据为空")
our_code_id: str | None = Field(None, description="我们后台配置的代码位 ID(104xxx);旧数据为空")
hour: int | None = Field(None, description="北京时间小时 023(granularity=hour 时有值;按天为 null)")
impressions: int = Field(..., description="展示条数(每条广告展示一条;轮播每条各计一次)")
revenue_yuan: float = Field(..., description="收益(元)= Σ(eCPM元 ÷ 1000);测试应用多为 0")
expected_coin: int = Field(..., description="应发金币(按公式复算,与金币审计同源)")
actual_coin: int = Field(..., description="实发金币(实际入账,按现发奖算法)")
matched: bool = Field(..., description="该组应发==实发(组内任一条不符则 false)")
adns: list[str] = Field(default_factory=list, description="实际填充的底层 ADN 子渠道集合(如 pangle/gdt)")
impression_records: list[AdRevenueImpression] = Field(
default_factory=list,
description="该组逐条展示明细(时间/eCPM/收益/adn);展开下钻用,无发奖也有(只要有展示)",
)
records: list[AdRevenueRecord] = Field(
default_factory=list,
description="该组逐条发奖复算明细(eCPM/因子1/份数/LT/因子2/应发/实发/一致);展开下钻用,纯展示无发奖记录的组为空",
)
class AdRevenueReportOut(BaseModel):
"""报表响应:全量统计 + 按天趋势 + 聚合明细。"""
date_from: str = Field(..., description="报表起始日期(北京时间 YYYY-MM-DD)")
date_to: str = Field(..., description="报表结束日期(北京时间 YYYY-MM-DD,闭区间;单日时与 date_from 相同)")
daily: list[AdRevenueDaily] = Field(..., description="按日期汇总序列(全量,供按天趋势图)")
total: int = Field(..., description="聚合组总数(全量,不受 limit 影响)")
truncated: bool = Field(..., description="明细是否被 limit 截断")
total_impressions: int = Field(..., description="全量展示条数合计")
total_revenue_yuan: float = Field(..., description="全量收益合计(元)")
total_expected_coin: int = Field(..., description="全量应发金币合计")
total_actual_coin: int = Field(..., description="全量实发金币合计")
mismatch_count: int = Field(..., description="应发≠实发的组数(=0 说明全部按公式发放)")
items: list[AdRevenueRow] = Field(..., description="聚合明细(按 用户→类型→代码位 排序)")
+1 -1
View File
@@ -10,7 +10,7 @@ class ConfigItemOut(BaseModel):
key: str key: str
label: str label: str
group: str group: str
type: str # int / int_list / dict_str_int type: str # int / int_list / dict_str_int / bool
help: str | None = None help: str | None = None
default: Any default: Any
value: Any value: Any
+122
View File
@@ -0,0 +1,122 @@
"""admin CPS 分发与对账 schemas。金额统一「分」(cents),前端 yuan() 展示。"""
from __future__ import annotations
from datetime import datetime
from pydantic import BaseModel, ConfigDict, Field
# ───────────── 群(sid) ─────────────
class CpsGroupOut(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: int
sid: str
name: str
member_count: int | None = None
status: str
remark: str | None = None
created_at: datetime
class CpsGroupCreate(BaseModel):
name: str = Field(min_length=1, max_length=128)
# 留空 → 后端自动生成 g<id>;填写则必须字母+数字(美团 sid 限制)。
sid: str | None = Field(default=None, max_length=64, pattern=r"^[A-Za-z0-9]+$")
member_count: int | None = Field(default=None, ge=0)
remark: str | None = Field(default=None, max_length=256)
class CpsGroupUpdate(BaseModel):
name: str | None = Field(default=None, min_length=1, max_length=128)
member_count: int | None = Field(default=None, ge=0)
status: str | None = Field(default=None, pattern=r"^(active|archived)$")
remark: str | None = Field(default=None, max_length=256)
# ───────────── 活动(actId) ─────────────
class CpsActivityOut(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: int
platform: str
name: str
act_id: str | None = None
product_view_sign: str | None = None
status: str
remark: str | None = None
created_at: datetime
class CpsActivityCreate(BaseModel):
name: str = Field(min_length=1, max_length=128)
platform: str = Field(default="meituan", pattern=r"^(meituan|taobao|jd)$")
act_id: str | None = Field(default=None, max_length=64)
product_view_sign: str | None = Field(default=None, max_length=128)
remark: str | None = Field(default=None, max_length=256)
# ───────────── 转链 ─────────────
class CpsReferralLinkRequest(BaseModel):
group_id: int
activity_id: int
# 链接类型:1长链 2短链 3deeplink。默认短链(微信可打开)+deeplink。
link_types: list[int] = Field(default_factory=lambda: [2, 3])
class CpsReferralLinkOut(BaseModel):
redirect_url: str # ★ 我们的群发短链(/c/{code}),发群就用这个,点击经我们统计
code: str
sid: str
group_name: str
activity_name: str
target_url: str # 实际 302 跳转的美团短链
link_map: dict[str, str] # 美团原始 1长链/2短链/3deeplink(参考/备用)
# ───────────── 对账拉单 ─────────────
class CpsReconcileResult(BaseModel):
fetched: int # 本次美团返回的订单条数
inserted: int # 新入库
updated: int # 已存在→更新状态/佣金
pages: int # 翻了几页
# ───────────── 订单明细 ─────────────
class CpsOrderOut(BaseModel):
model_config = ConfigDict(from_attributes=True)
id: int
order_id: str
sid: str | None = None
act_id: str | None = None
pay_price_cents: int | None = None
commission_cents: int | None = None
commission_rate: str | None = None
mt_status: str | None = None
invalid_reason: str | None = None
product_name: str | None = None
pay_time: datetime | None = None
# ───────────── 统计 ─────────────
class CpsGroupStat(BaseModel):
group_id: int | None = None # None = 未归群的 sid(历史/其它来源)
sid: str | None = None
name: str
member_count: int | None = None
click_pv: int = 0 # 点击总次数
click_uv: int = 0 # 独立点击(按 ip+ua 近似去重)
order_count: int # 有效订单数(不含取消/风控)
settled_count: int # 已结算单数
canceled_count: int # 取消+风控单数
gmv_cents: int # 成交额(有效单 payPrice 合计)
est_commission_cents: int # 预估佣金(有效单 profit 合计)
settled_commission_cents: int # 已结算佣金
class CpsStatsOut(BaseModel):
groups: list[CpsGroupStat]
total_order_count: int
total_est_commission_cents: int
total_settled_commission_cents: int
+32 -1
View File
@@ -17,7 +17,12 @@ from fastapi import APIRouter, Header
from app.api.deps import DbSession from app.api.deps import DbSession
from app.api.internal.price import _check_secret from app.api.internal.price import _check_secret
from app.repositories import store_mapping as repo from app.repositories import store_mapping as repo
from app.schemas.store_mapping import StoreMappingIn, StoreMappingOut from app.schemas.store_mapping import (
StoreMappingIn,
StoreMappingInvalidateIn,
StoreMappingInvalidateOut,
StoreMappingOut,
)
logger = logging.getLogger("shagua.internal.store") logger = logging.getLogger("shagua.internal.store")
@@ -77,3 +82,29 @@ def report_store_mapping(
payload.source_device_id, payload.source_user_id, payload.source_device_id, payload.source_user_id,
) )
return StoreMappingOut(inserted=created, row_id=row_id) return StoreMappingOut(inserted=created, row_id=row_id)
@router.post(
"/store-mapping/invalidate",
response_model=StoreMappingInvalidateOut,
summary="标记某平台 shopId 的缓存 deeplink 失效(pricebot 撞错误页回退时上报,lookup 不再返回)",
)
def invalidate_store_mapping(
payload: StoreMappingInvalidateIn,
db: DbSession,
x_internal_secret: Annotated[str | None, Header()] = None,
) -> StoreMappingInvalidateOut:
_check_secret(x_internal_secret)
if payload.platform == "taobao":
affected = repo.mark_taobao_deeplink_invalid(db, payload.shop_id)
elif payload.platform == "jd":
affected = repo.mark_jd_deeplink_invalid(db, payload.shop_id)
else:
# 当前只接淘宝/京东; 其它平台先 no-op(affected=0), 不报错 — 向后兼容 pricebot 将来扩展。
logger.info("store_mapping invalidate 跳过: platform=%s 暂不支持", payload.platform)
return StoreMappingInvalidateOut(ok=True, affected=0)
logger.info(
"store_mapping invalidate platform=%s shop_id=%s → 标记失效 %d",
payload.platform, payload.shop_id, affected,
)
return StoreMappingInvalidateOut(ok=True, affected=affected)
+36 -2
View File
@@ -32,6 +32,8 @@ from app.schemas.ad import (
FeedRewardIn, FeedRewardIn,
FeedRewardOut, FeedRewardOut,
PangleCallbackOut, PangleCallbackOut,
RewardNoShowIn,
RewardNoShowOut,
TestGrantIn, TestGrantIn,
TestGrantOut, TestGrantOut,
WatchReportIn, WatchReportIn,
@@ -85,7 +87,7 @@ def pangle_callback(request: Request, db: DbSession) -> PangleCallbackOut:
params = dict(request.query_params) params = dict(request.query_params)
if not pangle.verify_callback_sign(params, settings.PANGLE_REWARD_SECRET): if not pangle.verify_callback_sign_any(params, settings.pangle_reward_secrets):
logger.warning("pangle callback bad sign trans_id=%s", params.get("trans_id")) logger.warning("pangle callback bad sign trans_id=%s", params.get("trans_id"))
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="bad sign") raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="bad sign")
@@ -242,10 +244,12 @@ def ecpm_report(payload: EcpmReportIn, user: CurrentUser, db: DbSession) -> Ecpm
ad_type=payload.ad_type, ecpm_raw=payload.ecpm, ad_type=payload.ad_type, ecpm_raw=payload.ecpm,
ad_session_id=payload.ad_session_id, ad_session_id=payload.ad_session_id,
adn=payload.adn, slot_id=payload.slot_id, adn=payload.adn, slot_id=payload.slot_id,
app_env=payload.app_env, our_code_id=payload.our_code_id,
) )
logger.info( logger.info(
"ad ecpm report user_id=%d type=%s session=%s ecpm=%s adn=%s slot=%s", "ad ecpm report user_id=%d type=%s session=%s ecpm=%s adn=%s slot=%s app=%s code=%s",
user.id, payload.ad_type, payload.ad_session_id, payload.ecpm, payload.adn, payload.slot_id, user.id, payload.ad_type, payload.ad_session_id, payload.ecpm, payload.adn, payload.slot_id,
payload.app_env, payload.our_code_id,
) )
return EcpmReportOut(ok=True) return EcpmReportOut(ok=True)
@@ -359,6 +363,9 @@ def feed_reward(payload: FeedRewardIn, user: CurrentUser, db: DbSession) -> Feed
ad_session_id=payload.ad_session_id, ad_session_id=payload.ad_session_id,
adn=payload.adn, adn=payload.adn,
slot_id=payload.slot_id, slot_id=payload.slot_id,
app_env=payload.app_env,
our_code_id=payload.our_code_id,
aborted=payload.aborted,
) )
logger.info( logger.info(
"feed ad reward user_id=%d event=%s status=%s units=%d coin=%d", "feed ad reward user_id=%d event=%s status=%s units=%d coin=%d",
@@ -371,3 +378,30 @@ def feed_reward(payload: FeedRewardIn, user: CurrentUser, db: DbSession) -> Feed
unit_count=rec.unit_count, unit_count=rec.unit_count,
daily_limit=rewards.get_ad_daily_limit(db), daily_limit=rewards.get_ad_daily_limit(db),
) )
@router.post(
"/reward-noshow",
response_model=RewardNoShowOut,
summary="激励视频提前关闭/未发奖留痕",
dependencies=[Depends(rate_limit(120, 60, "ad-reward-noshow"))],
)
def reward_noshow(payload: RewardNoShowIn, user: CurrentUser, db: DbSession) -> RewardNoShowOut:
"""激励视频展示了但用户提前关/跳过、未触发 S2S 发奖时,客户端 best-effort 上报一条留痕,
让广告收益报表能呈现「有展示、没发金币」的原因。不发金币;同一 session 已发奖则跳过。
"""
rec = crud_ad.record_reward_noshow(
db,
user.id,
ad_session_id=payload.ad_session_id,
ecpm=payload.ecpm,
adn=payload.adn,
slot_id=payload.slot_id,
app_env=payload.app_env,
our_code_id=payload.our_code_id,
)
logger.info(
"ad reward noshow user_id=%d session=%s watched=%ds -> status=%s",
user.id, payload.ad_session_id, payload.watched_seconds, rec.status,
)
return RewardNoShowOut(ok=True, status=rec.status)
+14 -1
View File
@@ -18,7 +18,7 @@ import httpx
from fastapi import APIRouter, HTTPException, Request, status from fastapi import APIRouter, HTTPException, Request, status
from fastapi.concurrency import run_in_threadpool from fastapi.concurrency import run_in_threadpool
from app.api.deps import DbSession from app.api.deps import CurrentUser, DbSession
from app.core.config import settings from app.core.config import settings
from app.core.pricebot_router import pick_pricebot from app.core.pricebot_router import pick_pricebot
from app.db.session import SessionLocal from app.db.session import SessionLocal
@@ -28,6 +28,7 @@ from app.schemas.coupon_state import (
CouponPromptDismissIn, CouponPromptDismissIn,
CouponPromptShouldShowOut, CouponPromptShouldShowOut,
CouponPromptShownIn, CouponPromptShownIn,
CouponStatsOut,
) )
logger = logging.getLogger("shagua.coupon") logger = logging.getLogger("shagua.coupon")
@@ -268,3 +269,15 @@ def coupon_completed_today_reset(
与 /prompt/reset 配套:开发设置「重置今日领券弹窗状态」一键把今日状态全清。MVP 不鉴权。""" 与 /prompt/reset 配套:开发设置「重置今日领券弹窗状态」一键把今日状态全清。MVP 不鉴权。"""
coupon_repo.reset_today_completion(db, payload.device_id) coupon_repo.reset_today_completion(db, payload.device_id)
return {"ok": True} return {"ok": True}
@router.get(
"/stats",
response_model=CouponStatsOut,
summary="累计领券数(「我的」页战绩卡「领取优惠券 X 张」)",
)
def coupon_stats(user: CurrentUser, db: DbSession) -> CouponStatsOut:
"""该登录用户累计领到的券数(SUM(claimed_count),口径见 coupon_repo.sum_claimed_count)。
**鉴权(CurrentUser)**——区别于同文件不鉴权的 /step 透传与 /prompt 频控(那些按 device_id):
个人战绩按 user_id 聚合,必须有登录态。"""
return CouponStatsOut(coupon_count=coupon_repo.sum_claimed_count(db, user.id))
+39
View File
@@ -0,0 +1,39 @@
"""CPS 群发短链跳转:用户点 /c/{code} → 记一条点击 → 302 跳美团。
公网、无鉴权(群里任何人点都要能跳)。记点击失败绝不影响跳转(用户体验优先)。
"""
from __future__ import annotations
from fastapi import APIRouter, Depends, Request
from fastapi.responses import RedirectResponse
from sqlalchemy.orm import Session
from app.db.session import get_db
from app.repositories import cps_link as cps_link_repo
router = APIRouter(tags=["cps-redirect"])
# code 不存在/失效时的兜底落地(避免用户看到报错)
_FALLBACK_URL = "https://www.meituan.com/"
def _client_ip(request: Request) -> str | None:
xff = request.headers.get("x-forwarded-for")
if xff:
return xff.split(",")[0].strip()
return request.client.host if request.client else None
@router.get("/c/{code}", summary="群发短链跳转(记点击 + 302)")
def cps_redirect(code: str, request: Request, db: Session = Depends(get_db)):
link = cps_link_repo.get_by_code(db, code)
if link is None:
return RedirectResponse(_FALLBACK_URL, status_code=302)
try:
cps_link_repo.record_click(
db, link=link, ip=_client_ip(request),
ua=request.headers.get("user-agent"),
)
except Exception:
pass # 记点击失败不阻断跳转
return RedirectResponse(link.target_url, status_code=302)
+1 -1
View File
@@ -21,7 +21,7 @@ logger = logging.getLogger("shagua.feedback")
router = APIRouter(prefix="/api/v1/feedback", tags=["feedback"]) router = APIRouter(prefix="/api/v1/feedback", tags=["feedback"])
_MAX_IMAGES = 6 _MAX_IMAGES = 6
_CONTENT_MAX = 2000 _CONTENT_MAX = 200
_CONTACT_MAX = 128 _CONTACT_MAX = 128
+17 -1
View File
@@ -2,6 +2,7 @@
路由前缀 `/api/v1/platform`: 路由前缀 `/api/v1/platform`:
GET /stats 首页三统计(帮助用户 / 完成比价 / 累计节省),按运营后台配的模式算。 GET /stats 首页三统计(帮助用户 / 完成比价 / 累计节省),按运营后台配的模式算。
GET /flags 客户端运营 feature flag(比价/领券期广告开关等),客户端拉取后缓存。
展示模式(real/manual/random,每指标独立)与计算逻辑见 app/repositories/ops_stat.py。 展示模式(real/manual/random,每指标独立)与计算逻辑见 app/repositories/ops_stat.py。
""" """
@@ -12,9 +13,15 @@ import logging
from fastapi import APIRouter, Query from fastapi import APIRouter, Query
from app.api.deps import DbSession from app.api.deps import DbSession
from app.repositories import app_config
from app.repositories import ops_marquee as marquee_crud from app.repositories import ops_marquee as marquee_crud
from app.repositories import ops_stat as crud from app.repositories import ops_stat as crud
from app.schemas.platform import PlatformStatsOut, SavingsFeedItem, SavingsFeedOut from app.schemas.platform import (
AppFlagsOut,
PlatformStatsOut,
SavingsFeedItem,
SavingsFeedOut,
)
logger = logging.getLogger("shagua.platform") logger = logging.getLogger("shagua.platform")
@@ -35,3 +42,12 @@ def stats(db: DbSession) -> PlatformStatsOut:
def savings_feed(db: DbSession, limit: int = Query(8, ge=1, le=30)) -> SavingsFeedOut: def savings_feed(db: DbSession, limit: int = Query(8, ge=1, le=30)) -> SavingsFeedOut:
items = marquee_crud.get_feed(db, limit=limit) items = marquee_crud.get_feed(db, limit=limit)
return SavingsFeedOut(items=[SavingsFeedItem(**it) for it in items]) return SavingsFeedOut(items=[SavingsFeedItem(**it) for it in items])
@router.get("/flags", response_model=AppFlagsOut, summary="客户端运营 feature flag(不鉴权)")
def flags(db: DbSession) -> AppFlagsOut:
"""客户端拉取运营开关并缓存(app 启动 / 每场比价开始时刷新)。不鉴权:开关非敏感,
且比价无障碍服务取值时未必有登录态。值来自 app_config(admin 可改),空库回退默认。"""
return AppFlagsOut(
comparing_ad_enabled=bool(app_config.get_value(db, "comparing_ad_enabled")),
)
+30 -3
View File
@@ -136,8 +136,15 @@ class Settings(BaseSettings):
# ===== 穿山甲激励视频(服务端发奖回调)===== # ===== 穿山甲激励视频(服务端发奖回调)=====
# 看完激励视频后穿山甲服务器回调本服务发金币(S2S,客户端被破解也刷不到)。 # 看完激励视频后穿山甲服务器回调本服务发金币(S2S,客户端被破解也刷不到)。
# PANGLE_REWARD_SECRET 是穿山甲后台配置的"奖励校验密钥",验签用,从后台取到后填 .env。 # 穿山甲后台配置的"奖励校验密钥"(m-key),验签用。每个 GroMore 广告位 m-key 不同(后台各自
# 生成),但共用同一回调 URL → 多个位的 m-key 都要配上。验签时所有非空 m-key 逐个试、任一通过
# 即接受(见 pangle.verify_callback_sign_any / 下面的 pangle_reward_secrets)。
PANGLE_CALLBACK_ENABLED: bool = False PANGLE_CALLBACK_ENABLED: bool = False
# 推荐:每个激励位的 m-key 分开一行配,清晰不混淆(留空的忽略)。
PANGLE_REWARD_SECRET_TEST: str = "" # 测试应用 激励位 104099649
PANGLE_REWARD_SECRET_TEST_DEDICATED: str = "" # 测试应用 专属激励位 104127529
PANGLE_REWARD_SECRET_PROD: str = "" # 正式应用 激励位 104099389
# 旧用法:单个或逗号分隔的多个 m-key,仍兼容(会与上面三个命名项合并去重)。
PANGLE_REWARD_SECRET: str = "" PANGLE_REWARD_SECRET: str = ""
# ⚠️ 仅本地联调:打开后开放 POST /api/v1/ad/test-grant,让(已登录的)客户端在没部署公网、 # ⚠️ 仅本地联调:打开后开放 POST /api/v1/ad/test-grant,让(已登录的)客户端在没部署公网、
@@ -145,10 +152,23 @@ class Settings(BaseSettings):
# 它让客户端能自助发奖 = 绕过反作弊,**生产必须保持 False**(默认 False;只在本地 .env 设 true)。 # 它让客户端能自助发奖 = 绕过反作弊,**生产必须保持 False**(默认 False;只在本地 .env 设 true)。
AD_REWARD_TEST_GRANT_ENABLED: bool = False AD_REWARD_TEST_GRANT_ENABLED: bool = False
@property
def pangle_reward_secrets(self) -> list[str]:
"""汇总所有 m-key 成列表(去空白、去空项、去重保序)。验签时逐个试、任一通过即接受
(见 pangle.verify_callback_sign_any)。来源可混用:三个命名项 + 旧的逗号分隔 PANGLE_REWARD_SECRET。"""
raw = [
*self.PANGLE_REWARD_SECRET.split(","),
self.PANGLE_REWARD_SECRET_TEST,
self.PANGLE_REWARD_SECRET_TEST_DEDICATED,
self.PANGLE_REWARD_SECRET_PROD,
]
cleaned = [s.strip() for s in raw if s and s.strip()]
return list(dict.fromkeys(cleaned)) # 去重保序
@property @property
def pangle_callback_configured(self) -> bool: def pangle_callback_configured(self) -> bool:
"""回调开关打开且验签密钥已配,才接受发奖回调。""" """回调开关打开且至少配了一个验签密钥,才接受发奖回调。"""
return bool(self.PANGLE_CALLBACK_ENABLED and self.PANGLE_REWARD_SECRET) return bool(self.PANGLE_CALLBACK_ENABLED and self.pangle_reward_secrets)
# ===== Pricebot 上游 (领券/比价业务透传目标) ===== # ===== Pricebot 上游 (领券/比价业务透传目标) =====
# pricebot-backend 默认跑在 8000。/api/v1/coupon/step 会透传到这里的 /api/coupon/step # pricebot-backend 默认跑在 8000。/api/v1/coupon/step 会透传到这里的 /api/coupon/step
@@ -190,6 +210,13 @@ class Settings(BaseSettings):
# MVP 落地页就放 app-server 的 /media 静态目录下(dl.html)。 # MVP 落地页就放 app-server 的 /media 静态目录下(dl.html)。
INVITE_LANDING_URL: str = "https://app-api.shaguabijia.com/media/dl.html" INVITE_LANDING_URL: str = "https://app-api.shaguabijia.com/media/dl.html"
# ===== CPS 群发短链跳转 =====
# 群发券链接套一层我们的短链 /c/{code} 做点击统计后 302 跳美团。这里是写进 admin
# 生成链接里的对外跳转域名前缀。本地填 http://localhost:8770;生产**强烈建议用独立
# 域名**(大量群发会被微信封,别用主 API 域名连累整个 App)。留空 → 跳转端点用请求
# 的 Host 兜底拼绝对地址。
CPS_REDIRECT_BASE: str = ""
# ===== CORS ===== # ===== CORS =====
CORS_ALLOW_ORIGINS: str = "" CORS_ALLOW_ORIGINS: str = ""
+20 -1
View File
@@ -11,7 +11,7 @@ from typing import Any
from app.core import rewards as r from app.core import rewards as r
# type 约定(给前端渲染编辑控件用):int / int_list / dict_str_int # type 约定(给前端渲染编辑控件用):int / int_list / dict_str_int / bool
CONFIG_DEFS: dict[str, dict[str, Any]] = { CONFIG_DEFS: dict[str, dict[str, Any]] = {
"signin_rewards": { "signin_rewards": {
"default": list(r.SIGNIN_REWARDS), "label": "签到 7 天金币档位", "default": list(r.SIGNIN_REWARDS), "label": "签到 7 天金币档位",
@@ -65,4 +65,23 @@ CONFIG_DEFS: dict[str, dict[str, Any]] = {
"group": "签到", "type": "int", "group": "签到", "type": "int",
"help": "Day1-Day6 签到后看完激励视频额外发放的固定金币;Day7 不展示也不允许膨胀。", "help": "Day1-Day6 签到后看完激励视频额外发放的固定金币;Day7 不展示也不允许膨胀。",
}, },
"comparing_ad_enabled": {
"default": True, "label": "比价/领券期信息流广告",
"group": "看广告", "type": "bool",
"help": (
"开启后,比价进行中 + 领券等候期会在悬浮窗展示穿山甲信息流广告(变现行为);"
"关闭则全程不出广告。客户端按 app 启动 / 每场比价开始时拉取并缓存,故为「最终一致」的"
"远程开关(下一场比价生效),用于出问题时无需发版即可快速止血。debug 包可用本地开关覆盖。"
),
},
"withdraw_auto_reconcile_enabled": {
"default": True, "label": "提现自动对账",
"group": "钱包", "type": "bool",
"help": (
"开启后后台 worker 每隔一段时间自动扫描超时仍「打款中」的提现单并归一化"
"(查微信/退款/撤单)。需部署侧 env WITHDRAW_AUTO_RECONCILE_ENABLED=true 启动 worker "
"进程后此开关才起效;扫描间隔/超时阈值仍由 env 控制。关掉只停自动扫描,"
"提现页「批量对账」手动按钮不受影响。"
),
},
} }
+17 -4
View File
@@ -14,8 +14,11 @@ from sqlalchemy.exc import SQLAlchemyError
from app.core.config import settings from app.core.config import settings
from app.db.session import SessionLocal from app.db.session import SessionLocal
from app.integrations.wxpay import WxPayNotConfiguredError from app.integrations.wxpay import WxPayNotConfiguredError
from app.repositories import app_config
from app.repositories import wallet as wallet_repo from app.repositories import wallet as wallet_repo
_AUTO_RECONCILE_KEY = "withdraw_auto_reconcile_enabled"
logger = logging.getLogger("shagua.withdraw_reconcile") logger = logging.getLogger("shagua.withdraw_reconcile")
_LOCK_PATH = Path(__file__).resolve().parents[2] / "data" / "withdraw_reconcile.lock" _LOCK_PATH = Path(__file__).resolve().parents[2] / "data" / "withdraw_reconcile.lock"
@@ -59,8 +62,15 @@ def _single_instance_lock(stale_after_sec: int) -> Iterator[bool]:
_LOCK_PATH.unlink() _LOCK_PATH.unlink()
def _reconcile_once(older_than_minutes: int) -> dict: def _reconcile_once(older_than_minutes: int) -> dict | None:
"""读运营开关(app_config):关着返回 None(本轮跳过),开着才真扫单。
env WITHDRAW_AUTO_RECONCILE_ENABLED 是部署级总闸(决定 worker 起不起);
这里的 DB 开关是运营级日常开关,后台一改下一轮即生效、跨进程一致、无需重启。
"""
with SessionLocal() as db: with SessionLocal() as db:
if not app_config.get_value(db, _AUTO_RECONCILE_KEY):
return None
return wallet_repo.reconcile_pending_withdraws(db, older_than_minutes=older_than_minutes) return wallet_repo.reconcile_pending_withdraws(db, older_than_minutes=older_than_minutes)
@@ -78,16 +88,18 @@ async def _run_loop() -> None:
async def _run_locked_loop(interval: int, older_than: int) -> None: async def _run_locked_loop(interval: int, older_than: int) -> None:
logger.info( logger.info(
"withdraw auto reconcile started interval=%ss older_than=%sm", "withdraw auto reconcile worker started interval=%ss older_than=%sm "
"(runtime on/off via app_config '%s')",
interval, interval,
older_than, older_than,
_AUTO_RECONCILE_KEY,
) )
try: try:
while True: while True:
try: try:
_touch_lock() _touch_lock()
result = await asyncio.to_thread(_reconcile_once, older_than) result = await asyncio.to_thread(_reconcile_once, older_than)
if result["checked"] or result["resolved"]: if result is not None and (result["checked"] or result["resolved"]):
logger.info("withdraw auto reconcile result=%s", result) logger.info("withdraw auto reconcile result=%s", result)
except WxPayNotConfiguredError: except WxPayNotConfiguredError:
logger.warning("withdraw auto reconcile skipped: wxpay not configured") logger.warning("withdraw auto reconcile skipped: wxpay not configured")
@@ -103,7 +115,8 @@ async def _run_locked_loop(interval: int, older_than: int) -> None:
def start_withdraw_reconcile_worker() -> asyncio.Task | None: def start_withdraw_reconcile_worker() -> asyncio.Task | None:
if not settings.WITHDRAW_AUTO_RECONCILE_ENABLED: if not settings.WITHDRAW_AUTO_RECONCILE_ENABLED:
logger.info("withdraw auto reconcile disabled") # 部署级总闸关:worker 进程不启动(运营后台开关此时无效,需先在 env 打开)。
logger.info("withdraw auto reconcile worker not started (env master switch off)")
return None return None
if not settings.wxpay_configured: if not settings.wxpay_configured:
logger.warning("withdraw auto reconcile enabled but wxpay not configured") logger.warning("withdraw auto reconcile enabled but wxpay not configured")
+34
View File
@@ -0,0 +1,34 @@
# -*- coding: utf-8 -*-
"""美团城市字典(地级市)读取。
数据源:app/integrations/data/meituan_cities.json,由 tools/gen_meituan_cities.py 从
美团官方「城市字典」Excel 生成(359 个地级市:city_id / 城市名 / 省份)。
实测一个地级市 cityId 已覆盖其下辖县级市供给(徐州 → 邳州 / 新沂 / 睢宁 …),故无区县层级。
- ETL(scripts/pull_meituan_coupons.py)遍历 all_cities() 全量抓取入库。
- 读取侧将来「按城市过滤」时,也从这里取 city_id ↔ 城市名映射。
"""
from __future__ import annotations
import functools
import json
from pathlib import Path
_JSON = Path(__file__).resolve().parent / "data" / "meituan_cities.json"
@functools.lru_cache(maxsize=1)
def all_cities() -> list[dict]:
"""全部城市 [{city_id, name, province}, ...](359 个地级市)。"""
return json.loads(_JSON.read_text(encoding="utf-8"))
@functools.lru_cache(maxsize=1)
def _by_id() -> dict[str, dict]:
return {c["city_id"]: c for c in all_cities()}
def city_name(city_id: str) -> str | None:
"""city_id → 城市名(查不到返回 None)。"""
c = _by_id().get(city_id)
return c["name"] if c else None
File diff suppressed because it is too large Load Diff
+39 -2
View File
@@ -126,13 +126,23 @@ def query_coupon(
def get_referral_link( def get_referral_link(
*, *,
product_view_sign: str, product_view_sign: str | None = None,
act_id: str | None = None,
platform: int = 1, platform: int = 1,
biz_line: int | None = None, biz_line: int | None = None,
sid: str | None = None, sid: str | None = None,
link_type_list: list[int] | None = None, link_type_list: list[int] | None = None,
) -> dict[str, Any]: ) -> dict[str, Any]:
body: dict[str, Any] = {"productViewSign": product_view_sign} """换推广链接。actId(活动物料)与 productViewSign(商品券)二选一。
用 actId 入参时,出参 data 为 null,链接全在 referralLinkMap(实测)。
"""
if not act_id and not product_view_sign:
raise MeituanCpsError("act_id 或 product_view_sign 必填其一")
body: dict[str, Any] = {}
if act_id:
body["actId"] = act_id
else:
body["productViewSign"] = product_view_sign
if platform == 2: if platform == 2:
body["platform"] = 2 body["platform"] = 2
if biz_line is not None: if biz_line is not None:
@@ -142,3 +152,30 @@ def get_referral_link(
body["linkTypeList"] = link_type_list or [1, 3] body["linkTypeList"] = link_type_list or [1, 3]
return _call("/cps_open/common/api/v1/get_referral_link", body) return _call("/cps_open/common/api/v1/get_referral_link", body)
def query_order(
*,
sid: str | None = None,
start_time: int,
end_time: int,
query_time_type: int = 1,
page: int = 1,
limit: int = 100,
) -> dict[str, Any]:
"""按时间窗(+可选 sid)拉 CPS 订单做对账。实测要点:
- 分页是 page / limit(不是 pageNo/pageSize);startTime/endTime 是 10 位秒级时间戳
- query_time_type: 1 按支付时间 2 按更新时间
- 响应 data.dataList[],每条含 orderId / sid / payPrice(元) / profit(元) /
commissionRate / status(2付款 3完成 4取消 5风控 6结算) / payTime(秒) / actId 等
"""
body: dict[str, Any] = {
"queryTimeType": query_time_type,
"startTime": start_time,
"endTime": end_time,
"page": page,
"limit": limit,
}
if sid:
body["sid"] = sid
return _call("/cps_open/common/api/v1/query_order", body)
+10
View File
@@ -42,3 +42,13 @@ def verify_callback_sign(params: dict[str, str], secret: str) -> bool:
got = params.get("sign") or "" got = params.get("sign") or ""
expect = build_sign(trans_id, secret) expect = build_sign(trans_id, secret)
return hmac.compare_digest(got, expect) return hmac.compare_digest(got, expect)
def verify_callback_sign_any(params: dict[str, str], secrets: list[str]) -> bool:
"""对**多个** m-key 逐个验签,任一通过即接受。
多广告位场景:每个 GroMore 广告位的 m-key 由后台各自生成、互不相同,但本服务用
同一个回调 URL 接所有位的回调。配置里放多个密钥(逗号分隔),回调到达时挨个试。
仍然安全:伪造者必须知道其中某个 m-key 才能算出合法 sign;空列表 → 一律失败。
"""
return any(verify_callback_sign(params, s) for s in secrets if s)
+3
View File
@@ -20,6 +20,7 @@ from app.api.v1.compare import router as compare_router
from app.api.v1.compare_milestone import router as compare_milestone_router from app.api.v1.compare_milestone import router as compare_milestone_router
from app.api.v1.compare_record import router as compare_record_router from app.api.v1.compare_record import router as compare_record_router
from app.api.v1.coupon import router as coupon_router from app.api.v1.coupon import router as coupon_router
from app.api.v1.cps_redirect import router as cps_redirect_router
from app.api.internal.price import router as internal_price_router from app.api.internal.price import router as internal_price_router
from app.api.internal.store import router as internal_store_router from app.api.internal.store import router as internal_store_router
from app.api.v1.feedback import router as feedback_router from app.api.v1.feedback import router as feedback_router
@@ -107,6 +108,8 @@ app.include_router(report_router)
app.include_router(internal_price_router) app.include_router(internal_price_router)
app.include_router(internal_store_router) app.include_router(internal_store_router)
app.include_router(platform_router) app.include_router(platform_router)
# CPS 群发短链跳转 /c/{code}(公网无鉴权:记点击 → 302 跳美团)
app.include_router(cps_redirect_router)
# 用户上传文件(头像)静态服务。生产可改由 nginx 直接 serve MEDIA_ROOT。 # 用户上传文件(头像)静态服务。生产可改由 nginx 直接 serve MEDIA_ROOT。
_media_root = Path(settings.MEDIA_ROOT) _media_root = Path(settings.MEDIA_ROOT)
+4
View File
@@ -6,6 +6,10 @@ from app.models.ad_watch_log import AdWatchLog # noqa: F401
from app.models.admin import AdminAuditLog, AdminUser # noqa: F401 from app.models.admin import AdminAuditLog, AdminUser # noqa: F401
from app.models.app_config import AppConfig # noqa: F401 from app.models.app_config import AppConfig # noqa: F401
from app.models.comparison import ComparisonRecord # noqa: F401 from app.models.comparison import ComparisonRecord # noqa: F401
from app.models.cps_activity import CpsActivity # noqa: F401
from app.models.cps_group import CpsGroup # noqa: F401
from app.models.cps_link import CpsClick, CpsLink # noqa: F401
from app.models.cps_order import CpsOrder # noqa: F401
from app.models.comparison_milestone import ComparisonMilestoneClaim # noqa: F401 from app.models.comparison_milestone import ComparisonMilestoneClaim # noqa: F401
from app.models.coupon_state import ( # noqa: F401 from app.models.coupon_state import ( # noqa: F401
CouponClaimRecord, CouponClaimRecord,
+5
View File
@@ -35,6 +35,11 @@ class AdEcpmRecord(Base):
adn: Mapped[str | None] = mapped_column(String(32), nullable=True) adn: Mapped[str | None] = mapped_column(String(32), nullable=True)
# 实际展示用的代码位(底层 mediation rit,非客户端配置位) # 实际展示用的代码位(底层 mediation rit,非客户端配置位)
slot_id: Mapped[str | None] = mapped_column(String(64), nullable=True) slot_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
# 我们的穿山甲应用环境:prod(傻瓜比价正式应用) / test(测试应用)。客户端按 AdConfig.useProductionApp 上报。
# 与底层 adn 不同:这是「我们用的是哪个 App」,adn 是「聚合后实际填充的子渠道」。旧数据为 NULL。
app_env: Mapped[str | None] = mapped_column(String(16), nullable=True)
# 我们在穿山甲后台配置的代码位 ID(AdConfig.feedCodeId/rewardCodeId 返回的 104xxx,**非** slot_id 的底层 rit)。旧数据为 NULL。
our_code_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
# 客户端上报的 eCPM 原始字符串(单位:分/千次展示,SDK getEcpm 原值,原样存) # 客户端上报的 eCPM 原始字符串(单位:分/千次展示,SDK getEcpm 原值,原样存)
ecpm_raw: Mapped[str] = mapped_column(String(32), nullable=False) ecpm_raw: Mapped[str] = mapped_column(String(32), nullable=False)
# 北京时间日期串 'YYYY-MM-DD',按它等值做"按天聚合"(不在 SQL 里做跨时区 date 比较) # 北京时间日期串 'YYYY-MM-DD',按它等值做"按天聚合"(不在 SQL 里做跨时区 date 比较)
+3
View File
@@ -30,6 +30,9 @@ class AdFeedRewardRecord(Base):
ecpm_raw: Mapped[str] = mapped_column(String(32), nullable=False) ecpm_raw: Mapped[str] = mapped_column(String(32), nullable=False)
adn: Mapped[str | None] = mapped_column(String(32), nullable=True) adn: Mapped[str | None] = mapped_column(String(32), nullable=True)
slot_id: Mapped[str | None] = mapped_column(String(64), nullable=True) slot_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
# 来源(广告收益报表用):我们的应用环境 prod/test + 我们配置的代码位 104xxx,由客户端 feed-reward 上报带上。旧数据为 NULL。
app_env: Mapped[str | None] = mapped_column(String(16), nullable=True)
our_code_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
coin: Mapped[int] = mapped_column(Integer, nullable=False, default=0) coin: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
status: Mapped[str] = mapped_column(String(16), nullable=False, default="granted") status: Mapped[str] = mapped_column(String(16), nullable=False, default="granted")
+4
View File
@@ -33,6 +33,10 @@ class AdRewardRecord(Base):
ad_session_id: Mapped[str | None] = mapped_column(String(64), index=True, nullable=True) ad_session_id: Mapped[str | None] = mapped_column(String(64), index=True, nullable=True)
# 本次发奖采用的 eCPM 原始值(回调自带或按 ad_session_id 匹配的客户端上报) # 本次发奖采用的 eCPM 原始值(回调自带或按 ad_session_id 匹配的客户端上报)
ecpm_raw: Mapped[str | None] = mapped_column(String(32), nullable=True) ecpm_raw: Mapped[str | None] = mapped_column(String(32), nullable=True)
# 来源(广告收益报表用):我们的应用环境 prod/test + 我们配置的代码位 104xxx。
# S2S 回调本身不带这俩,发奖时按 ad_session_id 匹配 ad_ecpm_record 回填(查不到为 NULL)。
app_env: Mapped[str | None] = mapped_column(String(16), nullable=True)
our_code_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
# 北京时间日期串 'YYYY-MM-DD',按它等值统计当日发奖次数 # 北京时间日期串 'YYYY-MM-DD',按它等值统计当日发奖次数
reward_date: Mapped[str] = mapped_column(String(10), index=True, nullable=False) reward_date: Mapped[str] = mapped_column(String(10), index=True, nullable=False)
# 穿山甲上报的奖励名(参考,不作发奖依据) # 穿山甲上报的奖励名(参考,不作发奖依据)
+34
View File
@@ -0,0 +1,34 @@
"""CPS 可推广活动池(cps_activity)。
运营预存常推的活动(券),生成链接时选它,省得每次记 actId。当前只接美团:
`act_id` = 美团联盟「我要推广-活动推广」第一列的物料 ID。`platform` 预留,接
淘宝/京东时复用本表(各自的活动标识)。
"""
from __future__ import annotations
from datetime import datetime
from sqlalchemy import DateTime, Integer, String, func
from sqlalchemy.orm import Mapped, mapped_column
from app.db.base import Base
class CpsActivity(Base):
__tablename__ = "cps_activity"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
platform: Mapped[str] = mapped_column(String(20), nullable=False, default="meituan") # meituan|taobao|jd
name: Mapped[str] = mapped_column(String(128), nullable=False)
# 美团活动物料 ID(actId);淘宝/京东接入后存各自活动标识。
act_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
# 美团商品券 productViewSign(与 act_id 二选一转链;多数活动用 act_id)。
product_view_sign: Mapped[str | None] = mapped_column(String(128), nullable=True)
status: Mapped[str] = mapped_column(String(20), nullable=False, default="active") # active | archived
remark: Mapped[str | None] = mapped_column(String(256), nullable=True)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), nullable=False
)
def __repr__(self) -> str: # pragma: no cover
return f"<CpsActivity id={self.id} platform={self.platform} name={self.name!r}>"
+32
View File
@@ -0,0 +1,32 @@
"""CPS 推广群(cps_group)。
每个微信群一条记录,`sid` 是美团联盟二级渠道追踪位(get_referral_link / query_order
共用):发券时塞进推广链接,订单回来按 sid 归群对账。美团限制 sid 仅字母+数字 ≤64。
"""
from __future__ import annotations
from datetime import datetime
from sqlalchemy import DateTime, Integer, String, func
from sqlalchemy.orm import Mapped, mapped_column
from app.db.base import Base
class CpsGroup(Base):
__tablename__ = "cps_group"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
# 每群唯一的渠道标识,建群时分配(留空则自动生成 g<id>)。字母+数字,≤64。
sid: Mapped[str] = mapped_column(String(64), unique=True, index=True, nullable=False)
name: Mapped[str] = mapped_column(String(128), nullable=False)
# 群人数,运营填,作转化率分母(可空)。
member_count: Mapped[int | None] = mapped_column(Integer, nullable=True)
status: Mapped[str] = mapped_column(String(20), nullable=False, default="active") # active | archived
remark: Mapped[str | None] = mapped_column(String(256), nullable=True)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), nullable=False
)
def __repr__(self) -> str: # pragma: no cover
return f"<CpsGroup id={self.id} sid={self.sid!r} name={self.name!r}>"
+52
View File
@@ -0,0 +1,52 @@
"""CPS 群发短链(cps_link)+ 点击事件(cps_click)。
群发券链接套一层我们自己的短链 `/c/{code}`:用户点 → 记一条 cps_click → 302 跳
target_url(美团短链,微信可打开)。据此统计点击 PV/UV(按群/活动/时段),配合 cps_order
的下单/佣金做"点击→下单→佣金"漏斗。group_id/sid 在 cps_click 里冗余一份,统计直接按
群聚合点击、不必 join cps_link。
"""
from __future__ import annotations
from datetime import datetime
from sqlalchemy import DateTime, Integer, String, func
from sqlalchemy.orm import Mapped, mapped_column
from app.db.base import Base
class CpsLink(Base):
__tablename__ = "cps_link"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
# 短码:发群链接 /c/{code} 的 code,全局唯一。
code: Mapped[str] = mapped_column(String(16), unique=True, index=True, nullable=False)
group_id: Mapped[int] = mapped_column(Integer, index=True, nullable=False)
activity_id: Mapped[int] = mapped_column(Integer, index=True, nullable=False)
sid: Mapped[str] = mapped_column(String(64), index=True, nullable=False)
platform: Mapped[str] = mapped_column(String(20), nullable=False, default="meituan")
# 302 跳转目标(美团短链,微信内可打开并唤起 App)。
target_url: Mapped[str] = mapped_column(String(1024), nullable=False)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), nullable=False
)
def __repr__(self) -> str: # pragma: no cover
return f"<CpsLink id={self.id} code={self.code!r} sid={self.sid!r}>"
class CpsClick(Base):
__tablename__ = "cps_click"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
link_id: Mapped[int] = mapped_column(Integer, index=True, nullable=False)
group_id: Mapped[int] = mapped_column(Integer, index=True, nullable=False) # 冗余,按群聚合快
sid: Mapped[str] = mapped_column(String(64), index=True, nullable=False) # 冗余
ip: Mapped[str | None] = mapped_column(String(64), nullable=True)
ua: Mapped[str | None] = mapped_column(String(512), nullable=True)
clicked_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), index=True, nullable=False
)
def __repr__(self) -> str: # pragma: no cover
return f"<CpsClick id={self.id} link_id={self.link_id} group_id={self.group_id}>"
+63
View File
@@ -0,0 +1,63 @@
"""CPS 对账订单(cps_order)。
从美团联盟 query_order 按时间窗拉回、按 sid 归群的订单明细。字段对齐 query_order
实测返回:
- payPrice / profit 是「元」字符串 → 入库统一转「分」(与全站口径一致)
- payTime / updateTime 是秒级时间戳 → 入库转 tz-aware datetime
- status: 2付款 3完成 4取消 5风控 6结算(取消/风控不计佣金)
order_id 全局唯一,reconcile 按它 upsert(订单状态会变,重复拉则更新)。
"""
from __future__ import annotations
from datetime import datetime
from sqlalchemy import JSON, DateTime, Integer, String, func
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.orm import Mapped, mapped_column
from app.db.base import Base
class CpsOrder(Base):
__tablename__ = "cps_order"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
# 美团订单号(加密串),全局唯一,upsert 幂等键。
order_id: Mapped[str] = mapped_column(String(64), unique=True, index=True, nullable=False)
# 渠道追踪位 = 群 sid(历史无 sid 订单为空)。按它归群聚合。
sid: Mapped[str | None] = mapped_column(String(64), index=True, nullable=True)
act_id: Mapped[str | None] = mapped_column(String(64), index=True, nullable=True)
biz_line: Mapped[int | None] = mapped_column(Integer, nullable=True) # 1=外卖
trade_type: Mapped[int | None] = mapped_column(Integer, nullable=True) # 1=cps 2=cpa
pay_price_cents: Mapped[int | None] = mapped_column(Integer, nullable=True)
commission_cents: Mapped[int | None] = mapped_column(Integer, nullable=True) # 预估佣金(profit)
commission_rate: Mapped[str | None] = mapped_column(String(16), nullable=True) # "300"=3% "10"=0.1%
refund_price_cents: Mapped[int | None] = mapped_column(Integer, nullable=True)
refund_profit_cents: Mapped[int | None] = mapped_column(Integer, nullable=True)
# 美团订单状态: 2付款 3完成 4取消 5风控 6结算
mt_status: Mapped[str | None] = mapped_column(String(8), index=True, nullable=True)
invalid_reason: Mapped[str | None] = mapped_column(String(128), nullable=True)
product_name: Mapped[str | None] = mapped_column(String(512), nullable=True)
pay_time: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), index=True, nullable=True
)
mt_update_time: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
raw: Mapped[dict] = mapped_column(
JSON().with_variant(JSONB(), "postgresql"), nullable=False, default=dict
)
first_seen: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), nullable=False
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), onupdate=func.now(), nullable=False
)
def __repr__(self) -> str: # pragma: no cover
return (
f"<CpsOrder id={self.id} order_id={self.order_id!r} "
f"sid={self.sid!r} status={self.mt_status} profit_cents={self.commission_cents}>"
)
+5
View File
@@ -86,6 +86,9 @@ class StoreMapping(Base):
taobao_share_url: Mapped[str | None] = mapped_column(String(256), nullable=True) # m.tb.cn 短链 taobao_share_url: Mapped[str | None] = mapped_column(String(256), nullable=True) # m.tb.cn 短链
taobao_resolved_url: Mapped[str | None] = mapped_column(Text, nullable=True) # 解析出的目标 URL(含 shopId) taobao_resolved_url: Mapped[str | None] = mapped_column(Text, nullable=True) # 解析出的目标 URL(含 shopId)
taobao_deeplink: Mapped[str | None] = mapped_column(Text, nullable=True) # 拼好的 et-store/search deeplink taobao_deeplink: Mapped[str | None] = mapped_column(Text, nullable=True) # 拼好的 et-store/search deeplink
# 淘宝 deeplink 失效标记:比价撞"页面出错了"降级页时被置(pricebot server→server invalidate),
# NULL=有效。lookup 反查过滤掉非 NULL 的淘宝候选,不再返回坏 deeplink(重搜会写新行覆盖)。
taobao_deeplink_invalid_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
# ===== 美团原料(同淘宝;dpurl.cn 短链 → 302 反查 poi_id_str → imeituan:// deeplink)===== # ===== 美团原料(同淘宝;dpurl.cn 短链 → 302 反查 poi_id_str → imeituan:// deeplink)=====
# ⚠️ poi_id_str 每次分享重新加密、非稳定主键(调研文档 §八), 故单列存"可复跳的一次性票据", # ⚠️ poi_id_str 每次分享重新加密、非稳定主键(调研文档 §八), 故单列存"可复跳的一次性票据",
@@ -102,6 +105,8 @@ class StoreMapping(Base):
jd_share_url: Mapped[str | None] = mapped_column(String(256), nullable=True) # 3.cn 短链 jd_share_url: Mapped[str | None] = mapped_column(String(256), nullable=True) # 3.cn 短链
jd_resolved_url: Mapped[str | None] = mapped_column(Text, nullable=True) # 反查出的目标 openapp.jdmobile:// deeplink jd_resolved_url: Mapped[str | None] = mapped_column(Text, nullable=True) # 反查出的目标 openapp.jdmobile:// deeplink
jd_deeplink: Mapped[str | None] = mapped_column(Text, nullable=True) # 拼好的 pages/search 店内搜索 deeplink jd_deeplink: Mapped[str | None] = mapped_column(Text, nullable=True) # 拼好的 pages/search 店内搜索 deeplink
# 京东 deeplink 失效标记(同 taobao_deeplink_invalid_at):撞"当前门店超出配送范围"页时被置,NULL=有效。
jd_deeplink_invalid_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
# 灵活字段兜底(免得加字段就迁移) # 灵活字段兜底(免得加字段就迁移)
attrs: Mapped[dict | None] = mapped_column(_JSON, nullable=True) attrs: Mapped[dict | None] = mapped_column(_JSON, nullable=True)
+4
View File
@@ -27,6 +27,10 @@ class User(Base):
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True) id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
phone: Mapped[str] = mapped_column(String(20), unique=True, index=True, nullable=False) phone: Mapped[str] = mapped_column(String(20), unique=True, index=True, nullable=False)
# 对外展示的账号 ID:11 位纯数字、首位非 1(与手机号天然区分——手机号都以 1 开头)、全局唯一、
# 创建时随机生成、不可变、不参与登录(登录仍走 phone)。生成见 repositories/user._gen_username。
username: Mapped[str] = mapped_column(String(11), unique=True, index=True, nullable=False)
# 注册渠道:jverify / sms。后续加 wechat / apple 时扩 # 注册渠道:jverify / sms。后续加 wechat / apple 时扩
register_channel: Mapped[str] = mapped_column(String(20), nullable=False, default="jverify") register_channel: Mapped[str] = mapped_column(String(20), nullable=False, default="jverify")
+9 -1
View File
@@ -23,8 +23,14 @@ def create_ecpm_record(
ad_session_id: str | None = None, ad_session_id: str | None = None,
adn: str | None = None, adn: str | None = None,
slot_id: str | None = None, slot_id: str | None = None,
app_env: str | None = None,
our_code_id: str | None = None,
) -> AdEcpmRecord: ) -> AdEcpmRecord:
"""落一条 eCPM 上报记录。report_date 用北京时间当天,供按天聚合。""" """落一条 eCPM 上报记录。report_date 用北京时间当天,供按天聚合。
app_env(prod/test)与 our_code_id(我们后台配置的 104xxx 代码位)供广告收益报表按
应用/代码位聚合;与 adn(实际填充子渠道)/slot_id(底层 rit)是两组不同口径。
"""
if ad_session_id: if ad_session_id:
existing = find_by_session(db, user_id=user_id, ad_session_id=ad_session_id) existing = find_by_session(db, user_id=user_id, ad_session_id=ad_session_id)
if existing is not None: if existing is not None:
@@ -35,6 +41,8 @@ def create_ecpm_record(
ad_session_id=ad_session_id, ad_session_id=ad_session_id,
adn=adn, adn=adn,
slot_id=slot_id, slot_id=slot_id,
app_env=app_env,
our_code_id=our_code_id,
ecpm_raw=ecpm_raw, ecpm_raw=ecpm_raw,
report_date=cn_today().isoformat(), report_date=cn_today().isoformat(),
) )
+53 -4
View File
@@ -69,12 +69,19 @@ def grant_feed_reward(
ad_session_id: str | None = None, ad_session_id: str | None = None,
adn: str | None = None, adn: str | None = None,
slot_id: str | None = None, slot_id: str | None = None,
app_env: str | None = None,
our_code_id: str | None = None,
aborted: bool = False,
) -> AdFeedRewardRecord: ) -> AdFeedRewardRecord:
"""完成一条信息流广告后结算奖励。client_event_id 幂等,同号重试不重复发。 """比价/领券一整场信息流(轮播多条)结束后结算奖励。client_event_id 幂等,同号重试不重复发。
一期 eCPM/时长均由客户端上报,故服务端两道硬闸防刷:时长钳到 FEED_MAX_DURATION_SECONDS 发奖规则:**比价全程不关广告才发**,金额按整场**总观看时长**折份(每 10 秒 1 份)。
限单事件份数,eCPM 在 rewards.calculate_ad_reward_coin 内钳到 AD_ECPM_MAX_FEN 限单份金额; - aborted=True(用户中途 ✕ 关闭):整场不发,记 status='closed_early' 留痕(原因可查)。
叠加每日 get_ad_daily_limit 条数上限,把单用户日产出锁进有限区间 - 总时长不足 10 秒(unit_count==0):记 status='too_short' 不发
- 命中当日条数上限:记 status='capped' 不发。
duration_seconds 是整场累计秒数。一期 eCPM/时长均由客户端上报,故服务端两道硬闸防刷:时长钳到
FEED_MAX_DURATION_SECONDS 限单场份数,eCPM 在 rewards.calculate_ad_reward_coin 内钳到
AD_ECPM_MAX_FEN 限单份金额;叠加每日 get_ad_daily_limit 条数上限,把单用户日产出锁进有限区间。
""" """
existing = _find_by_event(db, client_event_id) existing = _find_by_event(db, client_event_id)
if existing is not None: if existing is not None:
@@ -85,6 +92,25 @@ def grant_feed_reward(
safe_duration = max(0, min(duration_seconds, FEED_MAX_DURATION_SECONDS)) safe_duration = max(0, min(duration_seconds, FEED_MAX_DURATION_SECONDS))
unit_count = safe_duration // FEED_REWARD_UNIT_SECONDS unit_count = safe_duration // FEED_REWARD_UNIT_SECONDS
# 用户中途关闭广告:整场不发(全程不关才发),留一条 closed_early 记录原因。优先级最高。
if aborted:
rec = AdFeedRewardRecord(
client_event_id=client_event_id,
user_id=user_id,
reward_date=today,
duration_seconds=safe_duration,
unit_count=unit_count,
ad_session_id=ad_session_id,
ecpm_raw=ecpm,
adn=adn,
slot_id=slot_id,
app_env=app_env,
our_code_id=our_code_id,
coin=0,
status="closed_early",
)
return _commit_record(db, rec, client_event_id)
if _granted_today(db, user_id, today) >= rewards.get_ad_daily_limit(db): if _granted_today(db, user_id, today) >= rewards.get_ad_daily_limit(db):
rec = AdFeedRewardRecord( rec = AdFeedRewardRecord(
client_event_id=client_event_id, client_event_id=client_event_id,
@@ -96,11 +122,32 @@ def grant_feed_reward(
ecpm_raw=ecpm, ecpm_raw=ecpm,
adn=adn, adn=adn,
slot_id=slot_id, slot_id=slot_id,
app_env=app_env,
our_code_id=our_code_id,
coin=0, coin=0,
status="capped", status="capped",
) )
return _commit_record(db, rec, client_event_id) return _commit_record(db, rec, client_event_id)
# 整场总时长不足 10 秒,凑不满一份 → 不发,记 too_short 留痕。
if unit_count == 0:
rec = AdFeedRewardRecord(
client_event_id=client_event_id,
user_id=user_id,
reward_date=today,
duration_seconds=safe_duration,
unit_count=0,
ad_session_id=ad_session_id,
ecpm_raw=ecpm,
adn=adn,
slot_id=slot_id,
app_env=app_env,
our_code_id=our_code_id,
coin=0,
status="too_short",
)
return _commit_record(db, rec, client_event_id)
coin = _unit_reward_total(db, user_id, ecpm, unit_count) coin = _unit_reward_total(db, user_id, ecpm, unit_count)
if coin > 0: if coin > 0:
crud_wallet.grant_coins( crud_wallet.grant_coins(
@@ -118,6 +165,8 @@ def grant_feed_reward(
ecpm_raw=ecpm, ecpm_raw=ecpm,
adn=adn, adn=adn,
slot_id=slot_id, slot_id=slot_id,
app_env=app_env,
our_code_id=our_code_id,
coin=coin, coin=coin,
status="granted", status="granted",
) )
+62 -4
View File
@@ -90,6 +90,16 @@ def grant_ad_reward(
today = cn_today().isoformat() today = cn_today().isoformat()
# 按 ad_session_id 匹配客户端 eCPM 上报:既用于缺 eCPM 时回退取值,也把「来源」
# (我们的应用 app_env + 我们配置的代码位 our_code_id)回填到发奖记录,供广告收益报表聚合。
# S2S 回调本身不带这俩;查不到(未上报 eCPM)则留空。
ecpm_rec = (
crud_ecpm.find_by_session(db, user_id=user_id, ad_session_id=ad_session_id)
if ad_session_id else None
)
src_app_env = ecpm_rec.app_env if ecpm_rec is not None else None
src_code_id = ecpm_rec.our_code_id if ecpm_rec is not None else None
# #3 每日上限:当前产品只保留发奖次数上限(默认 500 次)。旧的观看时长闸保留字段, # #3 每日上限:当前产品只保留发奖次数上限(默认 500 次)。旧的观看时长闸保留字段,
# 但 DAILY_AD_WATCH_SECONDS_LIMIT=0 时视为停用,不能命中 capped。 # 但 DAILY_AD_WATCH_SECONDS_LIMIT=0 时视为停用,不能命中 capped。
over_time = ( over_time = (
@@ -102,19 +112,18 @@ def grant_ad_reward(
trans_id=trans_id, user_id=user_id, coin=0, status="capped", trans_id=trans_id, user_id=user_id, coin=0, status="capped",
reward_date=today, reward_name=reward_name, raw=raw, reward_date=today, reward_name=reward_name, raw=raw,
reward_scene=reward_scene, ad_session_id=ad_session_id, ecpm_raw=ecpm, reward_scene=reward_scene, ad_session_id=ad_session_id, ecpm_raw=ecpm,
app_env=src_app_env, our_code_id=src_code_id,
) )
return _commit_record(db, rec, trans_id) return _commit_record(db, rec, trans_id)
ecpm_raw = ecpm ecpm_raw = ecpm or (ecpm_rec.ecpm_raw if ecpm_rec is not None else None)
if not ecpm_raw and ad_session_id:
ecpm_rec = crud_ecpm.find_by_session(db, user_id=user_id, ad_session_id=ad_session_id)
ecpm_raw = ecpm_rec.ecpm_raw if ecpm_rec is not None else None
if not ecpm_raw: if not ecpm_raw:
rec = AdRewardRecord( rec = AdRewardRecord(
trans_id=trans_id, user_id=user_id, coin=0, status="ecpm_missing", trans_id=trans_id, user_id=user_id, coin=0, status="ecpm_missing",
reward_date=today, reward_name=reward_name, raw=raw, reward_date=today, reward_name=reward_name, raw=raw,
reward_scene=reward_scene, ad_session_id=ad_session_id, ecpm_raw=None, reward_scene=reward_scene, ad_session_id=ad_session_id, ecpm_raw=None,
app_env=src_app_env, our_code_id=src_code_id,
) )
return _commit_record(db, rec, trans_id) return _commit_record(db, rec, trans_id)
@@ -131,6 +140,55 @@ def grant_ad_reward(
trans_id=trans_id, user_id=user_id, coin=coin, status="granted", trans_id=trans_id, user_id=user_id, coin=coin, status="granted",
reward_date=today, reward_name=reward_name, raw=raw, reward_date=today, reward_name=reward_name, raw=raw,
reward_scene=reward_scene, ad_session_id=ad_session_id, ecpm_raw=ecpm_raw, reward_scene=reward_scene, ad_session_id=ad_session_id, ecpm_raw=ecpm_raw,
app_env=src_app_env, our_code_id=src_code_id,
)
return _commit_record(db, rec, trans_id)
def record_reward_noshow(
db: Session,
user_id: int,
*,
ad_session_id: str,
ecpm: str | None = None,
adn: str | None = None,
slot_id: str | None = None,
app_env: str | None = None,
our_code_id: str | None = None,
reward_scene: str = "reward_video",
) -> AdRewardRecord:
"""客户端上报「激励视频展示了但用户提前关/跳过、未触发发奖」,落一条 coin=0 status='closed_early'
记录,供广告收益报表把「不发金币的原因」也呈现出来(只留痕,不发币)。
幂等键 trans_id = 'noreward:{ad_session_id}'(每次展示唯一)。若同一 ad_session_id 已有 granted
记录(S2S 已发奖,正常路径),说明用户其实看完了 → 跳过不写、原样返回那条,避免与正常发奖重复。
app_env/our_code_id 由客户端直接带上(它本就持有);查不到 user 抛 UnknownUserError。
"""
if db.get(User, user_id) is None:
raise UnknownUserError
# 同一次展示已正常发奖 → 不再记 closed_early(防与 S2S granted 重复)
granted = db.execute(
select(AdRewardRecord)
.where(
AdRewardRecord.user_id == user_id,
AdRewardRecord.ad_session_id == ad_session_id,
AdRewardRecord.status == "granted",
)
.limit(1)
).scalar_one_or_none()
if granted is not None:
return granted
trans_id = f"noreward:{ad_session_id}"
existing = _find_by_trans(db, trans_id)
if existing is not None:
return existing
rec = AdRewardRecord(
trans_id=trans_id, user_id=user_id, coin=0, status="closed_early",
reward_date=cn_today().isoformat(), reward_name=None, raw=None,
reward_scene=reward_scene, ad_session_id=ad_session_id, ecpm_raw=ecpm,
app_env=app_env, our_code_id=our_code_id,
) )
return _commit_record(db, rec, trans_id) return _commit_record(db, rec, trans_id)
+27 -1
View File
@@ -9,7 +9,7 @@ import logging
from datetime import date, datetime from datetime import date, datetime
from zoneinfo import ZoneInfo from zoneinfo import ZoneInfo
from sqlalchemy import delete, select from sqlalchemy import delete, func, select
from sqlalchemy.exc import IntegrityError from sqlalchemy.exc import IntegrityError
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
@@ -206,3 +206,29 @@ def record_claims(
) )
return 0 return 0
return written return written
# ===== 累计领券数(「我的」页战绩卡「领取优惠券 X 张」)=====
def sum_claimed_count(db: Session, user_id: int) -> int:
"""该用户累计领到的优惠券张数。口径(2026-06-15 用户定):SUM(claimed_count) ——
各成功领券记录的 pricebot 展示张数(claimed_count 列,存的是 display_count)之和,
与领券完成时给用户看的「本次领了 N 张」同源。
- 只算 status ∈ {success, already_claimed}:already_claimed=今日已领过,协议里算「已领到」;
failed / skipped 不计。
- claimed_count 为 0 的保持 0:那是同 count_group 合并去重项(pricebot 只让一条出数),不重复计;
为 NULL 的兜底成 1(成功领到至少 1 张;实际 pricebot to_dict 恒下发 display_count,NULL 基本不出现)。
- 维度 user_id:登录态领的券才归入。登录前匿名领的(user_id 为空)不算(产品可接受)。
"""
total = db.execute(
select(
func.coalesce(
func.sum(func.coalesce(CouponClaimRecord.claimed_count, 1)), 0
)
).where(
CouponClaimRecord.user_id == user_id,
CouponClaimRecord.status.in_(("success", "already_claimed")),
)
).scalar_one()
return int(total or 0)
+82
View File
@@ -0,0 +1,82 @@
"""CPS 群发短链 + 点击 数据访问(用户侧跳转端点 + admin 生成/统计 共用)。
短码生成去掉易混字符;点击按 (ip, ua) 近似去重 UV。点击量级小、admin 低频,统计在
Python 侧聚合,跨 PG/SQLite 无方言坑(与 admin cps repo 同风格)。
"""
from __future__ import annotations
import secrets
from datetime import datetime
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.models.cps_link import CpsClick, CpsLink
# 去掉易混字符 0/O/1/l/I,运营/用户肉眼复制不易错
_CODE_ALPHABET = "abcdefghijkmnpqrstuvwxyzABCDEFGHJKLMNPQRSTUVWXYZ23456789"
def _gen_code(n: int = 8) -> str:
return "".join(secrets.choice(_CODE_ALPHABET) for _ in range(n))
def get_by_code(db: Session, code: str) -> CpsLink | None:
return db.execute(select(CpsLink).where(CpsLink.code == code)).scalar_one_or_none()
def create_link(
db: Session, *, group_id: int, activity_id: int, sid: str, target_url: str,
platform: str = "meituan", commit: bool = True,
) -> CpsLink:
"""生成唯一短码并存库。冲突重试 5 次,仍冲突用 12 位兜底(概率近 0)。"""
code = _gen_code()
for _ in range(5):
if get_by_code(db, code) is None:
break
code = _gen_code()
else:
code = _gen_code(12)
link = CpsLink(
code=code, group_id=group_id, activity_id=activity_id, sid=sid,
target_url=target_url, platform=platform,
)
db.add(link)
if commit:
db.commit()
db.refresh(link)
else:
db.flush()
return link
def record_click(db: Session, *, link: CpsLink, ip: str | None = None, ua: str | None = None) -> None:
db.add(CpsClick(
link_id=link.id, group_id=link.group_id, sid=link.sid,
ip=ip, ua=(ua[:500] if ua else None),
))
db.commit()
def click_stats_by_group(
db: Session, *, date_from: datetime | None = None, date_to: datetime | None = None,
) -> dict[int, dict]:
"""按 group_id 聚合点击 → {group_id: {"pv": N, "uv": M}}。uv 按 (ip, ua) 近似去重。"""
stmt = select(CpsClick)
if date_from is not None:
stmt = stmt.where(CpsClick.clicked_at >= date_from)
if date_to is not None:
stmt = stmt.where(CpsClick.clicked_at <= date_to)
clicks = list(db.execute(stmt).scalars().all())
agg: dict[int, dict] = {}
seen: dict[int, set] = {}
for c in clicks:
a = agg.setdefault(c.group_id, {"pv": 0, "uv": 0})
a["pv"] += 1
s = seen.setdefault(c.group_id, set())
key = (c.ip, c.ua)
if key not in s:
s.add(key)
a["uv"] += 1
return agg
+41 -1
View File
@@ -15,7 +15,7 @@ from __future__ import annotations
import logging import logging
import math import math
from sqlalchemy import select from sqlalchemy import func, select, update
from sqlalchemy.exc import IntegrityError from sqlalchemy.exc import IntegrityError
from sqlalchemy.orm import Session from sqlalchemy.orm import Session
@@ -107,6 +107,40 @@ def upsert(db: Session, payload: StoreMappingIn) -> tuple[int, int | None]:
return 0, existing.id return 0, existing.id
def mark_taobao_deeplink_invalid(db: Session, shop_id: str) -> int:
"""把所有 id_taobao=shop_id 的行标记淘宝 deeplink 失效(置 invalid_at=now)。返回本次新标记的行数。
按 shopId 标记**所有**行 —— 同一个坏 shopId(撞淘宝"页面出错了"降级页)可能散在多次比价的
多行里, 全标掉才能让后续 lookup 不再返回它。幂等: 已标记的行(invalid_at 非 NULL)跳过。"""
result = db.execute(
update(StoreMapping)
.where(
StoreMapping.id_taobao == shop_id,
StoreMapping.taobao_deeplink_invalid_at.is_(None),
)
.values(taobao_deeplink_invalid_at=func.now())
)
db.commit()
return result.rowcount or 0
def mark_jd_deeplink_invalid(db: Session, store_id: str) -> int:
"""把所有 id_jd=store_id 的行标记京东 deeplink 失效(置 invalid_at=now)。返回本次新标记的行数。
同 mark_taobao_deeplink_invalid:按 storeId 标记**所有**行(撞京东"当前门店超出配送范围"页),
全标掉才能让后续 lookup 不再返回它。幂等:已标记的行(invalid_at 非 NULL)跳过。"""
result = db.execute(
update(StoreMapping)
.where(
StoreMapping.id_jd == store_id,
StoreMapping.jd_deeplink_invalid_at.is_(None),
)
.values(jd_deeplink_invalid_at=func.now())
)
db.commit()
return result.rowcount or 0
# ============================================================ # ============================================================
# 缓存查询: 比价前按"源平台店名"反查已沉淀的各目标平台店铺 id, 命中就让 pricebot 直接 # 缓存查询: 比价前按"源平台店名"反查已沉淀的各目标平台店铺 id, 命中就让 pricebot 直接
# deeplink 跳店内搜索, 省掉"开平台→进店→分享反查"整段。 # deeplink 跳店内搜索, 省掉"开平台→进店→分享反查"整段。
@@ -161,6 +195,12 @@ def lookup_nearest(
if tgt == src_key: if tgt == src_key:
continue # 不返回源平台自己 continue # 不返回源平台自己
cands = [r for r in rows if getattr(r, id_attr)] cands = [r for r in rows if getattr(r, id_attr)]
if tgt == "taobao":
# 失效的淘宝 deeplink(撞过错误页被 invalidate)整条排除 = 当没缓存, pricebot 走正常搜店。
cands = [r for r in cands if r.taobao_deeplink_invalid_at is None]
elif tgt == "jd":
# 同上:失效的京东 deeplink(撞"当前门店超出配送范围"被 invalidate)整条排除。
cands = [r for r in cands if r.jd_deeplink_invalid_at is None]
if not cands: if not cands:
continue continue
best = _pick_best(cands, lat, lng) best = _pick_best(cands, lat, lng)
+48
View File
@@ -4,6 +4,8 @@
""" """
from __future__ import annotations from __future__ import annotations
import secrets
import string
from datetime import datetime, timezone from datetime import datetime, timezone
from sqlalchemy import select from sqlalchemy import select
@@ -12,6 +14,50 @@ from sqlalchemy.orm import Session
from app.models.user import User from app.models.user import User
# ===== 创建时分配的标识:用户名(对外展示账号 ID)+ 默认昵称 =====
# 用户名 = 11 位纯数字,首位 2-9(避前导 0、避 1 与手机号区分),后 10 位 0-9。
# 空间 8×10^10,创建时随机生成 + 查重去重,全局唯一、不可变、不参与登录。
_USERNAME_FIRST = "23456789"
_USERNAME_DIGITS = "0123456789"
_USERNAME_LEN = 11
# 默认昵称 = 9 位大小写字母 + 数字随机串(创建时给,用户可后续改;不要求唯一)。
_NICKNAME_ALPHABET = string.ascii_letters + string.digits
_NICKNAME_LEN = 9
def _gen_username() -> str:
"""随机一个 11 位纯数字、首位非 0/1 的用户名(不保证唯一,唯一性由 _gen_unique_username 兜)。"""
return secrets.choice(_USERNAME_FIRST) + "".join(
secrets.choice(_USERNAME_DIGITS) for _ in range(_USERNAME_LEN - 1)
)
def _gen_nickname() -> str:
"""随机一个 9 位字母+数字的默认昵称。"""
return "".join(secrets.choice(_NICKNAME_ALPHABET) for _ in range(_NICKNAME_LEN))
def get_user_by_username(db: Session, username: str) -> User | None:
return db.execute(
select(User).where(User.username == username)
).scalar_one_or_none()
def _gen_unique_username(db: Session) -> str:
"""生成一个库里尚不存在的用户名(仿 invite.ensure_code 的碰撞重试)。
8×10^10 空间下碰撞极罕见,查重 + 重试足够;并发下的残留碰撞由 username 唯一约束
兜底(commit 抛 IntegrityError,与现有 phone 并发同号同级别,概率 ~ 用户数/8e10)。
"""
for _ in range(8):
uname = _gen_username()
if get_user_by_username(db, uname) is None:
return uname
raise RuntimeError("生成用户名连续碰撞,请重试")
def get_user_by_id(db: Session, user_id: int) -> User | None: def get_user_by_id(db: Session, user_id: int) -> User | None:
return db.get(User, user_id) return db.get(User, user_id)
@@ -36,6 +82,8 @@ def upsert_user_for_login(
if user is None: if user is None:
user = User( user = User(
phone=phone, phone=phone,
username=_gen_unique_username(db),
nickname=_gen_nickname(),
register_channel=register_channel, register_channel=register_channel,
last_login_at=now, last_login_at=now,
) )
+49 -5
View File
@@ -57,6 +57,12 @@ class EcpmReportIn(BaseModel):
) )
adn: str | None = Field(None, description="实际投放 ADN(getSdkName),如 pangle") adn: str | None = Field(None, description="实际投放 ADN(getSdkName),如 pangle")
slot_id: str | None = Field(None, description="实际展示代码位(底层 mediation rit)") slot_id: str | None = Field(None, description="实际展示代码位(底层 mediation rit)")
app_env: str | None = Field(
None, max_length=16, description="我们的穿山甲应用环境:prod(傻瓜比价正式) / test(测试应用)"
)
our_code_id: str | None = Field(
None, max_length=64, description="我们后台配置的代码位 ID(AdConfig 的 104xxx,非底层 rit)"
)
class EcpmReportOut(BaseModel): class EcpmReportOut(BaseModel):
@@ -115,24 +121,62 @@ class TestGrantOut(BaseModel):
class FeedRewardIn(BaseModel): class FeedRewardIn(BaseModel):
"""信息流广告完成后结算奖励。 """比价/领券一整场信息流(轮播多条)结束后结算奖励。
每展示满 10 秒累计一份奖励,视频完成后一次性入账。client_event_id 用于客户端超时重试幂等。 规则:全程不关广告才发,金额按整场**总观看时长**折份(每 10 秒 1 份)。client_event_id 用于
客户端超时重试幂等。中途被用户关闭时传 aborted=True,整场不发(只记 closed_early)。
""" """
client_event_id: str = Field(..., min_length=8, max_length=64, description="客户端生成的幂等事件 id") client_event_id: str = Field(..., min_length=8, max_length=64, description="客户端生成的幂等事件 id")
ad_session_id: str | None = Field( ad_session_id: str | None = Field(
None, min_length=8, max_length=64, description="客户端生成的一次信息流广告会话 id" None, min_length=8, max_length=64, description="客户端生成的一次信息流广告会话 id"
) )
ecpm: str = Field(..., description="信息流广告 eCPM,按分/千次展示处理(SDK getEcpm 原值,非元)") ecpm: str = Field(..., description="信息流 eCPM(代表值,按分/千次展示处理;SDK getEcpm 原值,非元)")
duration_seconds: int = Field(..., ge=0, description="本条广告实际展示/播放秒数") duration_seconds: int = Field(..., ge=0, description="整场累计观看秒数(轮播各条相加)")
adn: str | None = Field(None, description="实际投放 ADN") adn: str | None = Field(None, description="实际投放 ADN")
slot_id: str | None = Field(None, description="实际展示代码位") slot_id: str | None = Field(None, description="实际展示代码位")
app_env: str | None = Field(
None, max_length=16, description="我们的穿山甲应用环境:prod(傻瓜比价正式) / test(测试应用)"
)
our_code_id: str | None = Field(
None, max_length=64, description="我们后台配置的代码位 ID(AdConfig 的 104xxx,非底层 rit)"
)
aborted: bool = Field(
False, description="用户中途 ✕ 关闭广告(未走完比价):整场不发,记 closed_early"
)
class FeedRewardOut(BaseModel): class FeedRewardOut(BaseModel):
granted: bool = Field(..., description="本次是否入账。达上限时 false") granted: bool = Field(..., description="本次是否入账。达上限时 false")
status: str = Field(..., description="granted / capped") status: str = Field(..., description="granted / capped / too_short / closed_early")
coin: int = Field(..., description="本次发放金币") coin: int = Field(..., description="本次发放金币")
unit_count: int = Field(..., description="按 10 秒折算出的奖励份数") unit_count: int = Field(..., description="按 10 秒折算出的奖励份数")
daily_limit: int = Field(..., description="每日信息流展示次数上限") daily_limit: int = Field(..., description="每日信息流展示次数上限")
class RewardNoShowIn(BaseModel):
"""激励视频展示了但用户提前关闭/跳过、未触发发奖——上报留痕(不发金币)。
供广告收益报表把「有展示、没发金币」的原因也记录下来。user_id 由 JWT 取,不在 body。
"""
ad_session_id: str = Field(
..., min_length=8, max_length=64, description="本次展示会话 id(与 ecpm 上报、S2S extra 一致)"
)
watched_seconds: int = Field(0, ge=0, description="关闭前已观看秒数(仅留痕参考,不入库)")
ecpm: str | None = Field(None, description="本次展示 eCPM(分/千次,SDK getEcpm 原值);可空")
adn: str | None = Field(None, description="实际投放 ADN")
slot_id: str | None = Field(None, description="实际展示代码位(底层 mediation rit)")
app_env: str | None = Field(
None, max_length=16, description="我们的穿山甲应用环境:prod / test"
)
our_code_id: str | None = Field(
None, max_length=64, description="我们后台配置的代码位 ID(104xxx)"
)
class RewardNoShowOut(BaseModel):
ok: bool = Field(..., description="是否处理成功(落库或幂等命中)")
status: str = Field(
..., description="closed_early(已留痕) / granted(该次其实已发奖,跳过未写)"
)
+2
View File
@@ -18,6 +18,8 @@ class UserOut(BaseModel):
model_config = ConfigDict(from_attributes=True) # 允许直接 UserOut.model_validate(orm_user) model_config = ConfigDict(from_attributes=True) # 允许直接 UserOut.model_validate(orm_user)
id: int id: int
# 对外展示的账号 ID:11 位纯数字、首位非 1、全局唯一、创建时分配、不可变。区别于登录用的 phone。
username: str
phone: str phone: str
nickname: str | None = None nickname: str | None = None
avatar_url: str | None = None avatar_url: str | None = None
+10
View File
@@ -39,3 +39,13 @@ class CouponCompletedTodayOut(BaseModel):
"""这台设备今天是否已跑完整轮领券(到 done 帧)。完成 → 首页「去领取」卡置灰。""" """这台设备今天是否已跑完整轮领券(到 done 帧)。完成 → 首页「去领取」卡置灰。"""
completed: bool completed: bool
class CouponStatsOut(BaseModel):
"""「我的」页战绩卡「领取优惠券 X 张」数据源:该登录用户累计领到的券数。
口径(2026-06-15 用户定):SUM(claimed_count) —— 各成功领券记录的 pricebot 展示张数之和,
与领券完成时给用户看的「本次领了 N 张」同源。详见 repositories.coupon_state.sum_claimed_count。
"""
coupon_count: int
+6
View File
@@ -22,3 +22,9 @@ class SavingsFeedItem(BaseModel):
class SavingsFeedOut(BaseModel): class SavingsFeedOut(BaseModel):
items: list[SavingsFeedItem] items: list[SavingsFeedItem]
class AppFlagsOut(BaseModel):
"""客户端拉取的运营 feature flag(不鉴权,登录前也能拉)。客户端缓存后按需读。"""
comparing_ad_enabled: bool # 比价/领券期是否展示信息流广告(远程 kill-switch)
+14
View File
@@ -60,3 +60,17 @@ class StoreMappingOut(BaseModel):
inserted: int inserted: int
row_id: int | None = None row_id: int | None = None
class StoreMappingInvalidateIn(BaseModel):
"""标记某平台某 shopId 的缓存 deeplink 失效(pricebot 撞错误页回退时上报)。"""
platform: str # 目前只支持 "taobao"
shop_id: str # 失效的店铺 id(淘宝 = shopId = id_taobao)
class StoreMappingInvalidateOut(BaseModel):
"""标记结果。affected = 本次新标记失效的行数(按 shopId 标记所有匹配行)。"""
ok: bool
affected: int
+2
View File
@@ -63,6 +63,7 @@
| 34 | `POST /api/v1/ad/test-grant` | Bearer | [详情](./ad-test-grant.md) | | 34 | `POST /api/v1/ad/test-grant` | Bearer | [详情](./ad-test-grant.md) |
| 35 | `POST /api/v1/ad/ecpm-report` | Bearer | [详情](./ad-ecpm-report.md) | | 35 | `POST /api/v1/ad/ecpm-report` | Bearer | [详情](./ad-ecpm-report.md) |
| 35a | `POST /api/v1/ad/feed-reward` | Bearer | [详情](./ad-feed-reward.md) | | 35a | `POST /api/v1/ad/feed-reward` | Bearer | [详情](./ad-feed-reward.md) |
| 35b | `POST /api/v1/ad/reward-noshow` | Bearer | [详情](./ad-reward-noshow.md)(激励视频提前关闭/未发奖留痕,只记原因不发币) |
| **用户资料**(前缀 `/api/v1/user` ||| | **用户资料**(前缀 `/api/v1/user` |||
| 35 | `PATCH /api/v1/user/profile` | Bearer | [详情](./user-profile.md) | | 35 | `PATCH /api/v1/user/profile` | Bearer | [详情](./user-profile.md) |
| 36 | `POST /api/v1/user/avatar` | Bearer | [详情](./user-avatar.md) | | 36 | `POST /api/v1/user/avatar` | Bearer | [详情](./user-avatar.md) |
@@ -104,6 +105,7 @@
| A26 | `POST /admin/api/marquee-seeds/bulk` | operator | [详情](./admin-marquee-seeds.md) | | A26 | `POST /admin/api/marquee-seeds/bulk` | operator | [详情](./admin-marquee-seeds.md) |
| A27 | `GET /admin/api/marquee-seeds/preview` | admin | [详情](./admin-marquee-seeds.md) | | A27 | `GET /admin/api/marquee-seeds/preview` | admin | [详情](./admin-marquee-seeds.md) |
| A28 | `GET /admin/api/ad-coin-audit` | admin | [详情](./admin-ad-coin-audit.md)(看广告金币公式复算对账,只读) | | A28 | `GET /admin/api/ad-coin-audit` | admin | [详情](./admin-ad-coin-audit.md)(看广告金币公式复算对账,只读) |
| A29 | `GET /admin/api/ad-revenue-report` | admin | [详情](./admin-ad-revenue-report.md)(广告收益报表:按用户/日期/类型/应用/代码位 聚合 条数/收益/金币,只读) |
| - | `GET /admin/api/health` | 无 | admin 健康检查(无单独文档) | | - | `GET /admin/api/health` | 无 | admin 健康检查(无单独文档) |
> ⚠️ 美团三个接口当前**无鉴权**,且 `referral-link``sid` 允许客户端传值覆盖默认渠道——见各接口"备注"。 > ⚠️ 美团三个接口当前**无鉴权**,且 `referral-link``sid` 允许客户端传值覆盖默认渠道——见各接口"备注"。
+6 -4
View File
@@ -7,11 +7,13 @@
| 字段 | 类型 | 必填 | 说明 | | 字段 | 类型 | 必填 | 说明 |
|---|---|---|---| |---|---|---|---|
| `ad_type` | str | 是 | 广告类型:`reward_video`(激励视频) / `draw`(Draw 信息流) 等 | | `ad_type` | str | 是 | 广告类型:`reward_video`(激励视频) / `feed`(信息流) / `draw`(历史 Draw 信息流) 等 |
| `ad_session_id` | str\|null | 否 | 客户端生成的广告会话 ID;需和穿山甲 `extra.ad_session_id` 一致,用于 S2S 缺 eCPM 时匹配 | | `ad_session_id` | str\|null | 否 | 客户端生成的广告会话 ID;激励视频与穿山甲 `extra.ad_session_id` 一致,用于 S2S 缺 eCPM 时匹配。**信息流轮播每条展示用各自独立 id**(不复用比价会话 id,否则 `uq_ad_ecpm_record_session` 去重只留一条) |
| `ecpm` | str | 是 | 穿山甲 `getShowEcpm().getEcpm()` 原始字符串,单位是**分/千次展示**(非元),后端 ÷100 转元参与金币公式 | | `ecpm` | str | 是 | 穿山甲 `getShowEcpm().getEcpm()` 原始字符串,单位是**分/千次展示**(非元),后端 ÷100 转元参与金币公式 |
| `adn` | str\|null | 否 | 实际投放 ADN(`getSdkName`),如 `pangle` | | `adn` | str\|null | 否 | 实际投放 ADN(`getSdkName`),如 `pangle`(聚合后实际填充的子渠道) |
| `slot_id` | str\|null | 否 | 实际展示代码位(底层 mediation rit,非客户端配置位) | | `slot_id` | str\|null | 否 | 实际展示代码位(底层 mediation rit,非客户端配置位) |
| `app_env` | str\|null | 否 | **我们的**穿山甲应用环境:`prod`(傻瓜比价正式) / `test`(测试应用) |
| `our_code_id` | str\|null | 否 | **我们后台配置的**代码位 ID(`AdConfig` 的 104xxx,**非** `slot_id` 的底层 rit) |
`user_id` 不在 body 里——由 JWT 取(Bearer),防伪造。 `user_id` 不在 body 里——由 JWT 取(Bearer),防伪造。
@@ -23,7 +25,7 @@
| `ok` | bool | 落库即 `true` | | `ok` | bool | 落库即 `true` |
## 说明 ## 说明
客户端在广告**展示后**(`onAdShow``getShowEcpm()`)调用,把本次展示的 eCPM 落库做**内部收益统计/对账**。 客户端在广告**展示后**(`onAdShow``getShowEcpm()`)调用,把本次展示的 eCPM 落库做**内部收益统计/对账**。激励视频每次展示上报一条;**信息流轮播每条展示各上报一条**(每条独立 `ad_session_id`),作为「广告收益报表」展示条数/收益的数据源(见 [admin-ad-revenue-report](./admin-ad-revenue-report.md))。`app_env` + `our_code_id` 供报表按「我们的应用 / 我们配置的代码位」聚合,与 `adn`/`slot_id`(底层填充渠道/rit)是两组不同口径。
- 普通激励视频发奖会先用 S2S 回调自带 `ecpm`;若缺失,再按 `ad_session_id` 读取本接口上报的 eCPM;两边都没有则不发并记录异常。 - 普通激励视频发奖会先用 S2S 回调自带 `ecpm`;若缺失,再按 `ad_session_id` 读取本接口上报的 eCPM;两边都没有则不发并记录异常。
- **best-effort**:客户端 fire-and-forget,但普通激励视频若 S2S 缺 eCPM,这条上报会成为发奖依据。 - **best-effort**:客户端 fire-and-forget,但普通激励视频若 S2S 缺 eCPM,这条上报会成为发奖依据。
+12 -9
View File
@@ -1,6 +1,6 @@
# POST /api/v1/ad/feed-reward — 信息流广告完成后结算金币 # POST /api/v1/ad/feed-reward — 信息流广告完成后结算金币
点位 2:比价等待 / 领券信息流广告。每展示满 10 秒累计一份奖励,视频完成后一次性入账 点位 2:比价等待 / 领券信息流广告(轮播多条)。**整场比价全程不关广告才发**,金额按整场**总观看时长**折份(每 10 秒 1 份),结束时一次性入账。用户中途 ✕ 关闭则整场不发
## 鉴权 ## 鉴权
@@ -12,24 +12,27 @@
|---|---|---:|---| |---|---|---:|---|
| `client_event_id` | string | 是 | 客户端生成的幂等事件 id,8-64 字符 | | `client_event_id` | string | 是 | 客户端生成的幂等事件 id,8-64 字符 |
| `ad_session_id` | string\|null | 否 | 客户端生成的一次信息流广告会话 id,用于对账/排查 | | `ad_session_id` | string\|null | 否 | 客户端生成的一次信息流广告会话 id,用于对账/排查 |
| `ecpm` | string | 是 | 本信息流广告 eCPM(穿山甲 getEcpm 原值),按“分/千次展示”处理(非元) | | `ecpm` | string | 是 | 本信息流 eCPM 代表值(穿山甲 getEcpm 原值),按“分/千次展示”处理(非元) |
| `duration_seconds` | int | 是 | 实际展示/播放秒数 | | `duration_seconds` | int | 是 | **整场累计观看秒数**(轮播各条相加) |
| `adn` | string\|null | 否 | 实际投放 ADN | | `adn` | string\|null | 否 | 实际投放 ADN(聚合后实际填充的子渠道) |
| `slot_id` | string\|null | 否 | 实际展示代码位 | | `slot_id` | string\|null | 否 | 实际展示代码位(底层 mediation rit) |
| `app_env` | string\|null | 否 | **我们的**应用环境:`prod`(傻瓜比价正式) / `test`(测试应用) |
| `our_code_id` | string\|null | 否 | **我们后台配置的**代码位 ID(104xxx,非底层 rit);供广告收益报表按代码位聚合金币 |
| `aborted` | bool | 否 | 用户中途 ✕ 关闭广告(未走完比价):整场不发,仅记 `closed_early`。默认 `false` |
## 响应 ## 响应
| 字段 | 类型 | 说明 | | 字段 | 类型 | 说明 |
|---|---|---| |---|---|---|
| `granted` | bool | 本次是否入账;达上限时为 `false` | | `granted` | bool | 本次是否入账;未发(任一非 granted 状态)时为 `false` |
| `status` | string | `granted` / `capped` | | `status` | string | `granted`(已发) / `capped`(当日次数超限) / `too_short`(整场总时长<10s 凑不满一份) / `closed_early`(用户中途关闭) |
| `coin` | int | 本次发放金币 | | `coin` | int | 本次发放金币;非 granted 为 0 |
| `unit_count` | int | 按 10 秒折算出的奖励份数 | | `unit_count` | int | 按 10 秒折算出的奖励份数 |
| `daily_limit` | int | 每日信息流展示次数上限,默认 500 | | `daily_limit` | int | 每日信息流展示次数上限,默认 500 |
## 计算口径 ## 计算口径
- 奖励份数:`duration_seconds // 10` - 奖励份数:`整场总时长 // 10`
- 单份奖励:`eCPM / 1000 × 因子1(eCPM 档) × 因子2(当天累计份序号) × 10000`,四舍五入为整数金币。 - 单份奖励:`eCPM / 1000 × 因子1(eCPM 档) × 因子2(当天累计份序号) × 10000`,四舍五入为整数金币。
- eCPM 档:`0-100=0.1``101-200=0.3``201-400=0.4``>400=0.6` - eCPM 档:`0-100=0.1``101-200=0.3``201-400=0.4``>400=0.6`
- LT 档:第 1 份 `2.0`,第 2 份 `1.5`,第 3 份 `1.3`,第 4-10 份 `1.1`,第 11 份及以后 `1.0` - LT 档:第 1 份 `2.0`,第 2 份 `1.5`,第 3 份 `1.3`,第 4-10 份 `1.1`,第 11 份及以后 `1.0`
+1 -1
View File
@@ -22,7 +22,7 @@ GroMore 以 GET 回调,关键参数:
| `ecpm` | string | 本次广告 eCPM(同上,可用于收益分析) | | `ecpm` | string | 本次广告 eCPM(同上,可用于收益分析) |
| `sign` | string | 签名,见下 | | `sign` | string | 签名,见下 |
**验签**:`sign = SHA256("{m-key}:{trans_id}")` 十六进制(只签 `trans_id`,其余参数不参与)。算法细节、m-key 来源、为什么这样设计 → 见集成文档 [integrations/pangle](../integrations/pangle.md)。 **验签**:`sign = SHA256("{m-key}:{trans_id}")` 十六进制(只签 `trans_id`,其余参数不参与)。多激励位共用同一回调 URL → 服务端把各位的 m-key 都配上,`verify_callback_sign_any` 逐个试、任一过即接受。算法细节、m-key 配置项(`PANGLE_REWARD_SECRET_TEST/_TEST_DEDICATED/_PROD`)、为什么这样设计 → 见集成文档 [integrations/pangle](../integrations/pangle.md)。
## 出参 ## 出参
响应 `200`,**响应体必须是 `{"is_verify": bool, "reason": int}`**GroMore 规范)。 响应 `200`,**响应体必须是 `{"is_verify": bool, "reason": int}`**GroMore 规范)。
+41
View File
@@ -0,0 +1,41 @@
# POST /api/v1/ad/reward-noshow — 激励视频提前关闭/未发奖留痕
激励视频**展示了但用户提前关闭/跳过、未触发 S2S 发奖**时,客户端 best-effort 上报一条留痕记录,
让运营后台「广告数据」能呈现「有展示、没发金币」的原因。**不发金币**。
## 鉴权
需要 Bearer token。`user_id` 由 JWT 取,不在 body。
## 请求体
| 字段 | 类型 | 必填 | 说明 |
|---|---|---:|---|
| `ad_session_id` | string | 是 | 本次展示会话 id(与 eCPM 上报、S2S `extra.ad_session_id` 一致)8-64 字符 |
| `watched_seconds` | int | 否 | 关闭前已观看秒数(仅留痕参考,**不入库**)。默认 0 |
| `ecpm` | string\|null | 否 | 本次展示 eCPM(穿山甲 getEcpm 原值,分/千次展示) |
| `adn` | string\|null | 否 | 实际投放 ADN(聚合后实际填充子渠道) |
| `slot_id` | string\|null | 否 | 实际展示代码位(底层 mediation rit) |
| `app_env` | string\|null | 否 | **我们的**应用环境:`prod`(傻瓜比价正式) / `test`(测试应用) |
| `our_code_id` | string\|null | 否 | **我们后台配置的**代码位 ID(104xxx,非底层 rit) |
## 响应
| 字段 | 类型 | 说明 |
|---|---|---|
| `ok` | bool | 是否处理成功(落库或幂等命中) |
| `status` | string | `closed_early`(已留痕) / `granted`(该次其实已发奖,跳过未写) |
## 说明
- 在 `onAdClose` 时若本次**未触发发奖**(无 `onRewardVerify`/`onRewardArrived`)调用。
- **幂等**:写入 `ad_reward_record``trans_id = noreward:{ad_session_id}`(每次展示唯一),重试不重复。
- **防与正常发奖重复**:若同一 `ad_session_id` 已有 `status='granted'` 记录(S2S 已发,说明用户其实看完了),
则跳过不写,返回 `status='granted'`
- best-effort:客户端 fire-and-forget,失败只 log,不影响主流程。
- 信息流的「用户中途关闭」走 [ad-feed-reward](./ad-feed-reward.md) 的 `aborted=true`(记 `closed_early`),不走本接口。
## 数据写入
- `ad_reward_record` 新增一行(`coin=0``status='closed_early'``reward_scene='reward_video'`)。
- **不**写 `coin_account` / `coin_transaction`(不发币)。
+108
View File
@@ -0,0 +1,108 @@
# Admin 广告收益报表
> 所属:Admin 组(前缀 `/admin/api/ad-revenue-report`) | 鉴权:Admin Bearer(任意已登录 admin,只读) | [← 返回 API 索引](./README.md)
**用户 × 日期 × 广告类型 × 我们的应用 × 我们的代码位** 聚合,回答「每个用户某天、每类广告(激励视频 / 信息流 / 历史 Draw)分别**看了多少条**、**收益多少**、按现算法**发了多少金币**、广告来自**哪个应用的哪个代码位**」。**纯只读**,不发币、不改数据,也**不改发奖逻辑**。
相关表:[ad_ecpm_record](../database/ad_ecpm_record.md)、[ad_reward_record](../database/ad_reward_record.md)、[ad_feed_reward_record](../database/ad_feed_reward_record.md)。
## 数据来源(三流合并,聚合键 = user × ad_type × app_env × our_code_id)
| 指标 | 来源表 | 口径 |
|---|---|---|
| 展示条数 `impressions` | `ad_ecpm_record` | 每行 = 客户端一次广告展示。激励视频每次展示上报一条;**信息流轮播每条展示各上报一条**(每条独立 `ad_session_id`) |
| 收益 `revenue_yuan` | `ad_ecpm_record` | `Σ(eCPM元 ÷ 1000)`,即每条展示预估收益累加(eCPM 原值是分,÷100 转元;÷1000 是每千次→单次)。**预估口径,非结算;测试应用多为 0** |
| 应发/实发金币 `expected_coin`/`actual_coin` | `ad_reward_record`(reward_video)+ `ad_feed_reward_record`(feed) | **复用金币审计逐条复算**(`ad_audit.audit_rows`,与正式发奖同一公式口径,不另写公式),按同维度求和;`matched = 应发==实发`。**只读复算,不改发奖** |
| 来源应用/代码位 `app_env`/`our_code_id` | 上述各表回填 | `prod`(傻瓜比价)/`test`(测试);代码位是**我们后台配的 104xxx**,非底层 rit |
| 底层渠道 `adns` | `ad_ecpm_record` | 实际填充的 ADN 子渠道集合(pangle/gdt/...),附加参考 |
展示与金币来自不同表,做**并集**:有展示无金币(用户中途关、未达发奖)、有金币无展示(未上报 eCPM)各自成行。
## GET /admin/api/ad-revenue-report — 聚合报表
- 入参(均 query,可选):
| 参数 | 类型 | 默认 | 说明 |
|---|---|---|---|
| `date_from` | string | 今天 | 起始日 北京时间 `YYYY-MM-DD` |
| `date_to` | string | =`date_from` | 结束日 北京时间 `YYYY-MM-DD`,**闭区间**;单日时与 `date_from` 相同 |
| `user_id` | int | 全部 | 只看某用户;不传=所有用户 |
| `ad_type` | string | 全部 | `reward_video` / `feed` / `draw`;不传=全部类型 |
| `granularity` | string | `day` | `day`=按天 / `hour`=按小时(聚合键再加北京时间小时 0–23);**区间>1 天建议用 day** |
| `limit` | int(1~1000) | 500 | **展示**明细组数(截断;`total`/`total_*`/`daily` 按全量统计不受影响) |
约束:`date_to` 不早于 `date_from`、区间最长 **92 天**、日期须 `YYYY-MM-DD`,否则 `422`
- 出参 `200`:`AdRevenueReportOut`
| 字段 | 类型 | 说明 |
|---|---|---|
| `date_from` / `date_to` | string | 报表起止日期(闭区间) |
| `daily` | `AdRevenueDaily[]` | 按日期汇总序列(全量,供按天趋势图;不受 `limit` 影响) |
| `total` | int | 聚合组**总数**(全量,不受 `limit` 影响) |
| `truncated` | bool | 明细是否被 `limit` 截断 |
| `total_impressions` | int | 全量展示条数合计 |
| `total_revenue_yuan` | float | 全量收益合计(元) |
| `total_expected_coin` | int | 全量应发金币合计 |
| `total_actual_coin` | int | 全量实发金币合计 |
| `mismatch_count` | int | 应发≠实发的组数(=0 说明全部按公式发放) |
| `items` | `AdRevenueRow[]` | 聚合明细(按 日期→用户→类型→代码位 排序) |
### AdRevenueDaily(`daily[]` — 按天趋势)
| 字段 | 类型 | 说明 |
|---|---|---|
| `date` | string | 北京时间 `YYYY-MM-DD` |
| `impressions` | int | 当天展示条数合计 |
| `revenue_yuan` | float | 当天预估收益合计(元) |
| `expected_coin` | int | 当天应发金币合计 |
| `actual_coin` | int | 当天实发金币合计 |
### AdRevenueRow(`items[]`)
| 字段 | 类型 | 说明 |
|---|---|---|
| `report_date` | string | 该组所属日期 北京时间 `YYYY-MM-DD` |
| `user_id` | int | |
| `ad_type` | string | `reward_video` / `feed` / `draw` |
| `app_env` | string \| null | 我们的应用:`prod`(傻瓜比价)/`test`(测试);旧数据为空 |
| `our_code_id` | string \| null | 我们配置的代码位 104xxx;旧数据为空 |
| `hour` | int \| null | 北京时间小时 023(`granularity=hour` 时有值;按天为 null) |
| `impressions` | int | 展示条数 |
| `revenue_yuan` | float | 收益(元),预估口径 |
| `expected_coin` | int | 应发金币(公式复算,与金币审计同源) |
| `actual_coin` | int | 实发金币(实际入账) |
| `matched` | bool | 该组应发==实发(组内任一条不符则 false) |
| `adns` | string[] | 底层填充 ADN 子渠道集合 |
| `impression_records` | `AdRevenueImpression[]` | 该组**逐条展示明细**(前端展开下钻);只要有展示就非空 |
| `records` | `AdRevenueRecord[]` | 该组**逐条发奖复算明细**(前端展开下钻);纯展示无发奖的组为空 |
### AdRevenueImpression(`items[].impression_records[]` — 展开「展示明细」)
| 字段 | 类型 | 说明 |
|---|---|---|
| `id` | int | ad_ecpm_record 主键 |
| `created_at` | datetime | |
| `ecpm` | string | 本次展示 eCPM 原始值(分/千次展示) |
| `revenue_yuan` | float | 本次展示预估收益(元)= eCPM元 ÷ 1000 |
| `adn` | string \| null | 实际填充 ADN 子渠道 |
| `slot_id` | string \| null | 底层 mediation rit(非我们配置的广告位 ID) |
### AdRevenueRecord(`items[].records[]` — 展开「发奖明细」)
还原金币审计的逐条列,与发奖同一复算口径。
| 字段 | 类型 | 说明 |
|---|---|---|
| `record_id` | int | 发奖记录主键 |
| `created_at` | datetime | |
| `status` | string | `granted` / `capped` / `ecpm_missing` |
| `ecpm` | string \| null | 本次采用的 eCPM 原始值 |
| `ecpm_factor` | float \| null | 因子1(eCPM 档);非 granted 为空 |
| `units` | int | 折算份数:激励视频恒 1;信息流 = 满 10 秒份数 |
| `lt_index_start` / `lt_index_end` | int \| null | 占用「账号累计第几份」的起止 |
| `lt_factor_start` / `lt_factor_end` | float \| null | 因子2(LT)起止值 |
| `expected_coin` | int | 应发金币 |
| `actual_coin` | int | 实发金币 |
| `matched` | bool | 该条复算与实发是否一致 |
## 说明与局限
- **展示 vs 发奖分离**:信息流轮播一会话可展示多条(都计入 `impressions`),但发奖仍按现规则(一会话发一次),`coin` 不因展示条数变化——这是有意设计(用户中途关只记展示不发奖)。
- **历史 Draw 不可拆**:迁移(Draw→普通信息流)前,Draw 发奖混在 `ad_feed_reward_record` 且无类型标记,金币侧统一记 `feed`;迁移后 Draw 不再产生新数据。展示侧 `ad_type` 由客户端上报区分,故 `draw` 桶基本为空。
- **来源字段从上线起齐全**:`app_env`/`our_code_id` 是本期新增列,历史记录为 NULL(报表来源列留空)。
- **收益是预估**:基于客户端上报的 eCPM,非穿山甲后台结算值;以后台报表为结算权威。
- **对账聚合级 + 逐条下钻**:行级 `matched` 给出该组(用户×类型×应用×代码位)应发是否==实发;**展开 `records` 即可看该组逐条明细**(eCPM/因子1/份数/LT/因子2/应发/实发/一致)定位到具体记录。独立逐条审计接口 [admin-ad-coin-audit](./admin-ad-coin-audit.md) 仍保留(同一复算口径,可全局按场景/只看不符筛选)。
+2 -2
View File
@@ -6,7 +6,7 @@
**multipart/form-data**: **multipart/form-data**:
| 字段 | 类型 | 必填 | 说明 | | 字段 | 类型 | 必填 | 说明 |
|---|---|---|---| |---|---|---|---|
| `content` | string | ✓ | 反馈正文,**1-2000 字**(strip 后) | | `content` | string | ✓ | 反馈正文,**1-200 字**(strip 后) |
| `contact` | string | ✗ | 联系方式(微信/QQ/手机号),**≤128 字**。原型改版后客户端已不再采集、不传该字段(后端默认空串);保留字段兼容旧端 | | `contact` | string | ✗ | 联系方式(微信/QQ/手机号),**≤128 字**。原型改版后客户端已不再采集、不传该字段(后端默认空串);保留字段兼容旧端 |
| `images` | file[] | ✗ | 截图,**最多 6 张**,每张走头像同款校验(JPEG/PNG/WebP,≤ 5 MB,魔数嗅探) | | `images` | file[] | ✗ | 截图,**最多 6 张**,每张走头像同款校验(JPEG/PNG/WebP,≤ 5 MB,魔数嗅探) |
@@ -29,7 +29,7 @@
> 不返回上传的 image URL——这是给运营后台看的,客户端通常不需要。 > 不返回上传的 image URL——这是给运营后台看的,客户端通常不需要。
## 错误码 ## 错误码
- `400` 内容为空 / 内容超 2000 字 / 联系方式超 128 字 / 图片超 6 张 / 单图非法(空/过大/格式不对) - `400` 内容为空 / 内容超 200 字 / 联系方式超 128 字 / 图片超 6 张 / 单图非法(空/过大/格式不对)
- `401` 未带 token / token 无效或过期 / 用户被禁用 - `401` 未带 token / token 无效或过期 / 用户被禁用
- `422``content` 字段 - `422``content` 字段
+6 -4
View File
@@ -7,17 +7,19 @@
## 用在哪 / 增删改查 ## 用在哪 / 增删改查
- **C(插入)**:`POST /ad/ecpm-report`(`create_ecpm_record`)。每次广告展示上报一条;best-effort,丢一两条不影响业务。鉴权接口已确保 user 存在。 - **C(插入)**:`POST /ad/ecpm-report`(`create_ecpm_record`)。每次广告展示上报一条;best-effort,丢一两条不影响业务。鉴权接口已确保 user 存在。
- **U / D**:无。 - **U / D**:无。
- **R**:内部收益统计/对账(按 `(user_id, report_date)` 聚合);`count_today` 排查辅助。当前无面向 C 端用户的读接口。 - **R**:内部收益统计/对账(按 `(user_id, report_date)` 聚合);`count_today` 排查辅助;广告收益报表 [admin-ad-revenue-report](../api/admin-ad-revenue-report.md) 的展示条数/收益数据源(按 `app_env`/`our_code_id` 聚合)。当前无面向 C 端用户的读接口。
## 字段 ## 字段
| 列 | 类型 | 约束 / 默认 | 说明(取值 / join) | | 列 | 类型 | 约束 / 默认 | 说明(取值 / join) |
|---|---|---|---| |---|---|---|---|
| `id` | Integer | PK, autoincrement | | | `id` | Integer | PK, autoincrement | |
| `user_id` | Integer | FK→user.id, index, NOT NULL | 归属用户 | | `user_id` | Integer | FK→user.id, index, NOT NULL | 归属用户 |
| `ad_type` | String(32) | NOT NULL | 广告类型,取值如 `reward_video`(激励视频)/ `draw`(Draw 信息流);各类型各自上报,不强行统一代码位 | | `ad_type` | String(32) | NOT NULL | 广告类型,取值如 `reward_video`(激励视频)/ `feed`(信息流)/ `draw`(历史 Draw 信息流);各类型各自上报,不强行统一代码位 |
| `ad_session_id` | String(64) | UNIQUE, index, nullable | 客户端广告会话 ID;用于普通激励视频 S2S 缺 `ecpm` 时匹配发奖 | | `ad_session_id` | String(64) | UNIQUE, index, nullable | 客户端广告会话 ID;激励视频用于 S2S 缺 `ecpm` 时匹配发奖。**信息流轮播每条展示用各自独立 id**(不复用比价会话 id,否则 UNIQUE 去重只留一条) |
| `adn` | String(32) | nullable | 实际投放 ADN(`getShowEcpm().getSdkName()`,如 `pangle`/`gdt`) | | `adn` | String(32) | nullable | 实际投放 ADN(`getShowEcpm().getSdkName()`,如 `pangle`/`gdt`;聚合后实际填充的子渠道) |
| `slot_id` | String(64) | nullable | 实际展示用代码位(底层 mediation rit,非客户端配置位) | | `slot_id` | String(64) | nullable | 实际展示用代码位(底层 mediation rit,非客户端配置位) |
| `app_env` | String(16) | nullable | **我们的**穿山甲应用环境:`prod`(傻瓜比价正式)/`test`(测试应用);旧数据 NULL。广告收益报表按它聚合「来源应用」 |
| `our_code_id` | String(64) | nullable | **我们后台配置的**代码位 ID(`AdConfig` 的 104xxx,**非** `slot_id` 底层 rit);旧数据 NULL |
| `ecpm_raw` | String(32) | NOT NULL | 客户端上报的 eCPM **原始串**(穿山甲 getEcpm 原值,单位**分/千次展示**);后端 ÷100 转元参与金币公式 | | `ecpm_raw` | String(32) | NOT NULL | 客户端上报的 eCPM **原始串**(穿山甲 getEcpm 原值,单位**分/千次展示**);后端 ÷100 转元参与金币公式 |
| `report_date` | String(10) | index, NOT NULL | 北京时间日期串 `YYYY-MM-DD`,按它做按天聚合 | | `report_date` | String(10) | index, NOT NULL | 北京时间日期串 `YYYY-MM-DD`,按它做按天聚合 |
| `created_at` | DateTime(tz) | server_default now(), index | 时间 | | `created_at` | DateTime(tz) | server_default now(), index | 时间 |
+7 -5
View File
@@ -11,13 +11,15 @@
| `ad_session_id` | String(64) | index, nullable | 客户端生成的一次信息流广告会话 id | | `ad_session_id` | String(64) | index, nullable | 客户端生成的一次信息流广告会话 id |
| `user_id` | Integer | FK → `user.id`, index, NOT NULL | 用户 | | `user_id` | Integer | FK → `user.id`, index, NOT NULL | 用户 |
| `reward_date` | String(10) | index, NOT NULL | 北京时间日期 `YYYY-MM-DD` | | `reward_date` | String(10) | index, NOT NULL | 北京时间日期 `YYYY-MM-DD` |
| `duration_seconds` | Integer | NOT NULL | 实际展示/播放秒数 | | `duration_seconds` | Integer | NOT NULL | 整场比价累计观看秒数(轮播各条相加) |
| `unit_count` | Integer | NOT NULL | `duration_seconds // 10` 得到的奖励份数 | | `unit_count` | Integer | NOT NULL | `duration_seconds // 10` 得到的奖励份数 |
| `ecpm_raw` | String(32) | NOT NULL | 客户端上报 eCPM 原始值 | | `ecpm_raw` | String(32) | NOT NULL | 客户端上报 eCPM 原始值 |
| `adn` | String(32) | nullable | 实际投放 ADN | | `adn` | String(32) | nullable | 实际投放 ADN(聚合后实际填充子渠道) |
| `slot_id` | String(64) | nullable | 实际展示代码位 | | `slot_id` | String(64) | nullable | 实际展示代码位(底层 mediation rit) |
| `coin` | Integer | NOT NULL | 实发金币,`capped` 时为 0 | | `app_env` | String(16) | nullable | 来源应用 `prod`(傻瓜比价)/`test`(测试),客户端 feed-reward 上报;旧数据 NULL。广告收益报表金币侧按它聚合 |
| `status` | String(16) | NOT NULL | `granted` / `capped` | | `our_code_id` | String(64) | nullable | 我们配置的代码位 104xxx,客户端上报;旧数据 NULL |
| `coin` | Integer | NOT NULL | 实发金币,非 `granted` 时为 0 |
| `status` | String(16) | NOT NULL | `granted`(已发) / `capped`(当日次数超限) / `too_short`(整场总时长<10s 凑不满一份) / `closed_early`(用户中途 ✕ 关闭,全程未看完) |
| `created_at` | DateTime | index, NOT NULL | 创建时间 | | `created_at` | DateTime | index, NOT NULL | 创建时间 |
## 约束 ## 约束
+6 -4
View File
@@ -5,7 +5,7 @@
每条 = 穿山甲一次**服务端激励回调**。`trans_id` 唯一做幂等键(穿山甲会重试,同号只处理一次)。`reward_scene` 区分普通激励视频、签到膨胀等场景;`reward_date`(北京时间日期串)给普通激励视频"每日上限"计数用。 每条 = 穿山甲一次**服务端激励回调**。`trans_id` 唯一做幂等键(穿山甲会重试,同号只处理一次)。`reward_scene` 区分普通激励视频、签到膨胀等场景;`reward_date`(北京时间日期串)给普通激励视频"每日上限"计数用。
## 用在哪 / 增删改查 ## 用在哪 / 增删改查
- **C(插入)**:`POST /ad/pangle-callback`(穿山甲 S2S,经 SHA256 验签;`grant_ad_reward` 或场景业务处理)或 `POST /ad/test-grant`(本地联调)。普通激励视频三道闸:① 验签不过 → API 层 403,不进库;② `trans_id` 已存在 → 原样返回不重复发;③ **当日发奖次数(`DAILY_AD_REWARD_LIMIT`,默认 500)到顶** → 记一行 `status='capped'``coin=0`、不发币。否则按 eCPM 公式发币。 - **C(插入)**:`POST /ad/pangle-callback`(穿山甲 S2S,经 SHA256 验签;`grant_ad_reward` 或场景业务处理)或 `POST /ad/test-grant`(本地联调)。普通激励视频三道闸:① 验签不过 → API 层 403,不进库;② `trans_id` 已存在 → 原样返回不重复发;③ **当日发奖次数(`DAILY_AD_REWARD_LIMIT`,默认 500)到顶** → 记一行 `status='capped'``coin=0`、不发币。否则按 eCPM 公式发币。另:`POST /ad/reward-noshow`(`record_reward_noshow`,Bearer)在用户提前关/未发奖时记一行 `status='closed_early'``coin=0` 留痕(同 session 已 granted 则跳过)。
- **U / D**:无。 - **U / D**:无。
- **R**:`GET /ad/reward-status`(看广告页:今日已发次数/上限、单次金币、本轮已看/冷却结束、今日已看时长/上限);审计/对账整表回溯。 - **R**:`GET /ad/reward-status`(看广告页:今日已发次数/上限、单次金币、本轮已看/冷却结束、今日已看时长/上限);审计/对账整表回溯。
@@ -13,13 +13,15 @@
| 列 | 类型 | 约束 / 默认 | 说明(取值 / join) | | 列 | 类型 | 约束 / 默认 | 说明(取值 / join) |
|---|---|---|---| |---|---|---|---|
| `id` | Integer | PK, autoincrement | | | `id` | Integer | PK, autoincrement | |
| `trans_id` | String(64) | UNIQUE, index, NOT NULL | 穿山甲交易号(幂等键)。**被 `coin_transaction.ref_id` 引用**(biz_type=reward_video/signin_boost 等) | | `trans_id` | String(64) | UNIQUE, index, NOT NULL | 穿山甲交易号(幂等键)。**被 `coin_transaction.ref_id` 引用**(biz_type=reward_video/signin_boost 等)`closed_early` 留痕记录无 S2S 交易号,用合成键 `noreward:{ad_session_id}` |
| `user_id` | Integer | FK→user.id, index, NOT NULL | 归属用户(回调 media_extra 带回;不存在抛 UnknownUserError) | | `user_id` | Integer | FK→user.id, index, NOT NULL | 归属用户(回调 media_extra 带回;不存在抛 UnknownUserError) |
| `reward_scene` | String(32) | NOT NULL, default `reward_video` | 奖励场景:`reward_video` 普通激励视频;`signin_boost` 签到膨胀 | | `reward_scene` | String(32) | NOT NULL, default `reward_video` | 奖励场景:`reward_video` 普通激励视频;`signin_boost` 签到膨胀 |
| `ad_session_id` | String(64) | index, nullable | 客户端广告会话 ID,来自 `extra.ad_session_id`;用于匹配 `ad_ecpm_record` | | `ad_session_id` | String(64) | index, nullable | 客户端广告会话 ID,来自 `extra.ad_session_id`;用于匹配 `ad_ecpm_record` |
| `ecpm_raw` | String(32) | nullable | 本次发奖采用的 eCPM 原始值;可来自 S2S `ecpm` 或客户端上报 | | `ecpm_raw` | String(32) | nullable | 本次发奖采用的 eCPM 原始值;可来自 S2S `ecpm` 或客户端上报 |
| `coin` | Integer | NOT NULL, default 0 | 实发金币;`capped`/`ecpm_missing`/业务不满足时为 0 | | `app_env` | String(16) | nullable | 来源应用 `prod`(傻瓜比价)/`test`(测试);S2S 不带,发奖时按 `ad_session_id` 匹配 `ad_ecpm_record` 回填,查不到 NULL。广告收益报表金币侧按它聚合 |
| `status` | String(16) | NOT NULL, default `granted` | 取值:`granted`(已发)/ `capped`(当日次数超限)/ `ecpm_missing`(缺 eCPM)/ `not_signed`/`already_boosted`/`last_day` | | `our_code_id` | String(64) | nullable | 我们配置的代码位 104xxx(同上回填) |
| `coin` | Integer | NOT NULL, default 0 | 实发金币;`capped`/`ecpm_missing`/`closed_early`/业务不满足时为 0 |
| `status` | String(16) | NOT NULL, default `granted` | 取值:`granted`(已发)/ `capped`(当日次数超限)/ `ecpm_missing`(缺 eCPM)/ `closed_early`(展示了但用户提前关/跳过,未发奖,客户端 reward-noshow 留痕)/ `not_signed`/`already_boosted`/`last_day` |
| `reward_date` | String(10) | index, NOT NULL | 北京时间日期串 `YYYY-MM-DD`,按它等值统计当日发奖次数 | | `reward_date` | String(10) | index, NOT NULL | 北京时间日期串 `YYYY-MM-DD`,按它等值统计当日发奖次数 |
| `reward_name` | String(64) | nullable | 穿山甲上报奖励名(参考,不作发奖依据) | | `reward_name` | String(64) | nullable | 穿山甲上报奖励名(参考,不作发奖依据) |
| `raw` | String(1024) | nullable | 回调原始参数(审计排查) | | `raw` | String(1024) | nullable | 回调原始参数(审计排查) |
+2 -2
View File
@@ -14,8 +14,8 @@
## 字段 ## 字段
| 列 | 类型 | 约束 / 默认 | 说明(取值 / join) | | 列 | 类型 | 约束 / 默认 | 说明(取值 / join) |
|---|---|---|---| |---|---|---|---|
| `key` | String(64) | **PK** | 配置标识,取值见 `config_schema.CONFIG_DEFS`:`signin_rewards` / `min_exchange_coin` / `withdraw_min_cents` / `withdraw_max_cents` / `task_rewards` / `record_milestones` / `ad_reward_coin` / `ad_daily_limit` / `ad_max_coin` / `ad_round_count` / `ad_cooldown_sec` | | `key` | String(64) | **PK** | 配置标识,取值见 `config_schema.CONFIG_DEFS`:`signin_rewards` / `min_exchange_coin` / `withdraw_min_cents` / `withdraw_max_cents` / `task_rewards` / `record_milestones` / `ad_reward_coin` / `ad_daily_limit` / `ad_max_coin` / `ad_round_count` / `ad_cooldown_sec` / `signin_boost_coin` / `withdraw_auto_reconcile_enabled` |
| `value` | JSON(PG: JSONB) | NOT NULL | 配置值,类型随 key(`int` / `int_list` 如签到 14 档 / `dict_str_int` 如 task_rewards) | | `value` | JSON(PG: JSONB) | NOT NULL | 配置值,类型随 key(`int` / `int_list` 如签到 14 档 / `dict_str_int` 如 task_rewards / `bool` 如 withdraw_auto_reconcile_enabled) |
| `updated_by_admin_id` | Integer | nullable | 最后修改的管理员 id(= `admin_user.id`,软引用,无 FK) | | `updated_by_admin_id` | Integer | nullable | 最后修改的管理员 id(= `admin_user.id`,软引用,无 FK) |
| `updated_at` | DateTime(tz) | server_default now(), onupdate now() | 最后修改时间 | | `updated_at` | DateTime(tz) | server_default now(), onupdate now() | 最后修改时间 |
+9 -3
View File
@@ -16,12 +16,18 @@
| 函数 | 说明 | | 函数 | 说明 |
|---|---| |---|---|
| `build_sign(trans_id, secret) -> str` | 计算 `SHA256("{secret}:{trans_id}")` hex。自验签测试 / 模拟回调脚本共用,保证两端一致 | | `build_sign(trans_id, secret) -> str` | 计算 `SHA256("{secret}:{trans_id}")` hex。自验签测试 / 模拟回调脚本共用,保证两端一致 |
| `verify_callback_sign(params, secret) -> bool` | 校验回调签名。密钥空 / 缺 `trans_id` 一律失败;比较用 `hmac.compare_digest` 定长比较防时序侧信道 | | `verify_callback_sign(params, secret) -> bool` | 用**单个** m-key 校验回调签名。密钥空 / 缺 `trans_id` 一律失败;比较用 `hmac.compare_digest` 定长比较防时序侧信道 |
| `verify_callback_sign_any(params, secrets) -> bool` | 用**一组** m-key 逐个验签、任一通过即接受(多激励位共用同一回调 URL 时用)。空列表 → 失败 |
## 配置(多激励位 = 多 m-key)
每个 GroMore 激励位的 m-key 由后台各自生成、互不相同,但本服务用**同一个回调 URL** 接所有位的回调,因此把用到的位的 m-key 都配上;验签时 `verify_callback_sign_any` 逐个试、任一过即接受(仍安全:伪造者须知道其中某个 m-key 才能造出合法 sign)。m-key 在后台「GroMore 聚合管理 → 搜广告位 ID → 编辑」处获取。`settings.pangle_reward_secrets` 把下列来源汇总去重:
## 配置
| 配置项 | 说明 | | 配置项 | 说明 |
|---|---| |---|---|
| `PANGLE_REWARD_SECRET` | m-key(安全密钥)。后台「GroMore 聚合管理 → 搜广告位 ID → 编辑」处获取 | | `PANGLE_REWARD_SECRET_TEST` | 测试应用 激励位 104099649 的 m-key |
| `PANGLE_REWARD_SECRET_TEST_DEDICATED` | 测试应用 专属激励位 104127529 的 m-key |
| `PANGLE_REWARD_SECRET_PROD` | 正式应用 激励位 104099389 的 m-key |
| `PANGLE_REWARD_SECRET` | 旧用法:单个或逗号分隔多个 m-key,仍兼容(与上面三个命名项合并去重) |
## 踩坑 ## 踩坑
- **别用联盟代码位那套**:联盟代码位层级用的是另一套 Security Key + `isValid` 响应体,与 GroMore 广告位层级不通用。我们走 GroMore,别接错。 - **别用联盟代码位那套**:联盟代码位层级用的是另一套 Security Key + `isValid` 响应体,与 GroMore 广告位层级不通用。我们走 GroMore,别接错。
+15 -2
View File
@@ -41,18 +41,31 @@
> 已确认走 **GroMore 广告位层级**回调(客户端 useMediation(true);规范 supportcenter/26240), > 已确认走 **GroMore 广告位层级**回调(客户端 useMediation(true);规范 supportcenter/26240),
> **不是**联盟代码位层级(5416)。验签算法 / 响应格式 / reward_amount 解析**代码已按 GroMore 规范实现** > **不是**联盟代码位层级(5416)。验签算法 / 响应格式 / reward_amount 解析**代码已按 GroMore 规范实现**
> (2026-05-27),剩下的是后台配置 + 填密钥。 > (2026-05-27),剩下的是后台配置 + 填密钥。
>
> 🔴 **2026-06-16 实测的头号阻塞:穿山甲打线上后端会超时**。真机看广告时 SDK 日志
> `onRewardArrived valid=false errorCode=50002`,errorMsg =
> `Get "https://app-api.shaguabijia.com/api/v1/ad/pangle-callback...": context deadline exceeded`
> 而我们自己 curl 线上是秒回。说明**穿山甲机房到线上后端这条网络路径有问题**(疑似 nginx/云防火墙对
> 机房/境外 IP 限流或拦截)。**m-key 配对了也救不了——穿山甲根本没把请求送达。** 上线前必须单独验
> 「线上 `app-api.shaguabijia.com` 对穿山甲机房可达且响应够快」,否则用户看完发不了奖。
>
> ✅ **多激励位已支持**(2026-06-16):内部测试期 测试位/专属位/正式位 并存,m-key 各配一行
> (`PANGLE_REWARD_SECRET_TEST/_TEST_DEDICATED/_PROD`),`verify_callback_sign_any` 逐个试。本地已验通。
- [ ] **GroMore 后台配回调**:GroMore 聚合管理 → 搜广告位ID → 编辑 → 勾选「服务端激励回调」→ - [ ] **GroMore 后台配回调**:GroMore 聚合管理 → 搜广告位ID → 编辑 → 勾选「服务端激励回调」→
回调 URL 填 `https://app-api.shaguabijia.com/api/v1/ad/pangle-callback`(用域名,别用 IP)。 回调 URL 填 `https://app-api.shaguabijia.com/api/v1/ad/pangle-callback`(用域名,别用 IP)。
⚠️ 广告位层级配了就**别再在代码位层级重复配**(会导致发奖出问题)。 ⚠️ 广告位层级配了就**别再在代码位层级重复配**(会导致发奖出问题)。
- [ ] **拿 m-key(安全密钥)**:就在上面"编辑广告位"页获取,填到生产 `.env` `PANGLE_REWARD_SECRET` - [ ] **拿 m-key(安全密钥)**:就在上面"编辑广告位"页获取(每个激励位各一把),填到生产 `.env`命名变量
(正式位填 `PANGLE_REWARD_SECRET_PROD`;内部测试期 测试位/专属位 也并存就填 `_TEST`/`_TEST_DEDICATED`)。
(注:这是 GroMore 广告位的 m-key,**不是**联盟代码位那个 Security Key。) (注:这是 GroMore 广告位的 m-key,**不是**联盟代码位那个 Security Key。)
- [x] ~~换验签~~:`app/integrations/pangle.py` 已实现 `sign = SHA256("{m-key}:{trans_id}")`(GroMore 真实算法)。 - [x] ~~换验签~~:`app/integrations/pangle.py` 已实现 `sign = SHA256("{m-key}:{trans_id}")`(GroMore 真实算法)。
- [x] ~~响应格式~~:已返回 GroMore 要求的 `{"is_verify": bool, "reason": int}` - [x] ~~响应格式~~:已返回 GroMore 要求的 `{"is_verify": bool, "reason": int}`
- [x] ~~reward_amount~~:回调按 `reward_amount` 发金币(`rewards.resolve_ad_reward_coin`,带回退/夹紧); - [x] ~~reward_amount~~:回调按 `reward_amount` 发金币(`rewards.resolve_ad_reward_coin`,带回退/夹紧);
**后台广告位"奖励数量"须配成与 `AD_REWARD_COIN`(=100)一致**,保证"广告内展示/进度预告/到账"三者一致。 **后台广告位"奖励数量"须配成与 `AD_REWARD_COIN`(=100)一致**,保证"广告内展示/进度预告/到账"三者一致。
- [x] ~~透传 user_id~~:客户端已 `setUserID(userId)` + `setMediaExtra("uid:...")` - [x] ~~透传 user_id~~:客户端已 `setUserID(userId)` + `setMediaExtra("uid:...")`
- [ ] 生产 `.env`:`PANGLE_CALLBACK_ENABLED=true` + `PANGLE_REWARD_SECRET=<m-key>`(配齐才不返 503) - [ ] 生产 `.env`:`PANGLE_CALLBACK_ENABLED=true` + 至少一个 m-key 命名变量(配齐才不返 503)+ 确认
`AD_REWARD_TEST_GRANT_ENABLED=false`(绕过反作弊,生产红线)。改 `.env` 后**重启 uvicorn**
(watchfiles 不监听 .env)。验证:外部 curl 回调 URL 从 `503``403 bad sign` 即生效。
## C. 部署 + 包名 ## C. 部署 + 包名
+186 -52
View File
@@ -1,45 +1,74 @@
"""美团 CPS 券定时抓取入库(北京试点)。 """美团 CPS 券定时抓取入库(全国 359 个地级市)。
3 路券抓进 meituan_coupon ,销量/佣金排序从库里捞本地排序,不再实时打美团: 每个城市的 3 路券抓进 meituan_coupon ,智能推荐 / 销量最高从库里捞本地排序,
不再实时打美团:
1. search_waimai : 到家/外卖, 外卖 翻到尽头 1. search_waimai : 到家/外卖, 外卖 翻到尽头
2. search_meishi : 到家/外卖, 美食 翻到尽头 2. search_meishi : 到家/外卖, 美食 翻到尽头
3. store_supply : 到店, 多业务线供给(到餐+到综+酒店+门票) 翻到尽头 3. store_supply : 到店, 多业务线供给(到餐+到综+酒店+门票) 翻到尽头
城市来自 app/integrations/data/meituan_cities.json(美团官方城市字典,359 地级市)
实测一个地级市 cityId 已覆盖其下辖县级市(徐州 邳州/新沂),故按地级市抓即可
并发:城市级并发(每城内部仍顺序跑 3 ),主线程串行入库(Session 不跨线程)
实测单城全量 ~110s/~2300 ;15 并发抓完全国一轮 ~5060min,402 占比 ~3% 且退避全消化
mentor 要求每 3h 全量一轮,窗口充裕,故默认 12 并发 + 启动错峰,把瞬时峰值与 402 压更低
(source, product_view_sign) upsert 存最新态;last_seen 每轮刷新带文件锁,防止 (source, product_view_sign) upsert 存最新态;last_seen 每轮刷新带文件锁,防止
上一轮没跑完下一轮又起(本地 5~10min跨进程 cron 都安全) 上一轮没跑完下一轮又起
用法: 用法:
# 单轮(打通验证 / 给 cron 用,线上每 1h 一次) # 单轮全量(给 cron 用,线上每 3h 一次)
python -m scripts.pull_meituan_coupons --once python -m scripts.pull_meituan_coupons --once
# 本地循环(默认每 10min 一轮) # 常驻循环(每 3h 一轮)
python -m scripts.pull_meituan_coupons --loop --interval 600 python -m scripts.pull_meituan_coupons --loop --interval 10800
# 本地测试:只抓前 N 个城市 / 指定城市
python -m scripts.pull_meituan_coupons --once --limit 3
python -m scripts.pull_meituan_coupons --once --city-ids OCZOBCJDEXKE7KBN3BD7AYQG2Q
部署(服务器):推荐 cron --once( 3h),避免长驻进程孤儿:
0 */3 * * * cd /path/to/app-server && .venv/bin/python -m scripts.pull_meituan_coupons --once >> data/etl.log 2>&1
""" """
from __future__ import annotations from __future__ import annotations
import argparse import argparse
import hashlib import hashlib
import logging
import os import os
import re import re
import sys import sys
import tempfile import tempfile
import threading
import time import time
from concurrent.futures import ThreadPoolExecutor, as_completed
from datetime import datetime, timedelta, timezone from datetime import datetime, timedelta, timezone
# Windows 控制台按 UTF-8 输出中文/¥ # Windows 控制台按 UTF-8 输出中文/¥;line_buffering=True 让 print 每行即时 flush——
# 否则 stdout 重定向到 cron/后台日志文件时是块缓冲,要攒到 ~4KB 或进程退出才落盘,
# 常驻(--loop)时几乎看不到每轮进度。
try: try:
sys.stdout.reconfigure(encoding="utf-8") # type: ignore[attr-defined] sys.stdout.reconfigure(encoding="utf-8", line_buffering=True) # type: ignore[attr-defined]
except Exception: # noqa: BLE001 except Exception: # noqa: BLE001
pass pass
from sqlalchemy import delete, func, select from sqlalchemy import delete, func, select # noqa: E402
from sqlalchemy.dialects.postgresql import insert as pg_insert from sqlalchemy.dialects.postgresql import insert as pg_insert # noqa: E402
from app.db.session import SessionLocal from app.db.session import SessionLocal, engine # noqa: E402
from app.integrations.meituan import MeituanCpsError, _call from app.integrations.cities import all_cities # noqa: E402
from app.models.meituan_coupon import MeituanCoupon from app.integrations.meituan import MeituanCpsError, _call # noqa: E402
from app.models.meituan_coupon import MeituanCoupon # noqa: E402
# dev 默认 create_engine(echo=True) 会逐条打 SQL,本脚本逐城 upsert 会把日志刷爆;
# ETL 不需要 SQL 日志,显式关掉(入库不受影响;线上 prod 本就 echo=False)。
engine.echo = False
# 美团调用偶发错误(本机走代理高并发时的 SSL EOF / 上游 code=5 等)会被 meituan._call 的
# logger.exception 打完整 traceback,并发抓取下单轮可刷数十 KB 日志。ETL 自己用 _STATS 统计
# 「放弃页数」,无需逐条 traceback,故把美团 logger 压到 CRITICAL(线上直连少见此类错误)。
logging.getLogger("shagua.meituan").setLevel(logging.CRITICAL)
CITY_BEIJING = "WKV2HMXUEK634WP64CUCUQGM64"
QUERY_PATH = "/cps_open/common/api/v1/query_coupon" QUERY_PATH = "/cps_open/common/api/v1/query_coupon"
PAGE_SIZE = 20 PAGE_SIZE = 20
MAX_PAGES = 80 # 单路安全上限(搜索 ~52 页、供给 ~70 页) MAX_PAGES = 80 # 单路安全上限(搜索 ~52 页、供给 ~70 页)
@@ -49,7 +78,12 @@ RETRY = 7
# 无 sudo 部署时 data/ 常属 root,cps 写不了会导致每轮 PermissionError、cron 抓不进数据。 # 无 sudo 部署时 data/ 常属 root,cps 写不了会导致每轮 PermissionError、cron 抓不进数据。
# 需要指定位置时用环境变量 MEITUAN_ETL_LOCK 覆盖。 # 需要指定位置时用环境变量 MEITUAN_ETL_LOCK 覆盖。
LOCK_FILE = os.environ.get("MEITUAN_ETL_LOCK") or os.path.join(tempfile.gettempdir(), "meituan_etl.lock") LOCK_FILE = os.environ.get("MEITUAN_ETL_LOCK") or os.path.join(tempfile.gettempdir(), "meituan_etl.lock")
LOCK_STALE_SEC = 30 * 60 # 锁超过 30min 视为陈旧(进程异常退出残留),自动接管 # 多城全量一轮 ~50–60min,锁陈旧阈值放宽到 90min,避免把「正在跑的轮次」误判为残留而接管
LOCK_STALE_SEC = 90 * 60
DEFAULT_CONCURRENCY = 12 # 并发城市数(实测 15 并发 402 占 3% 可退避消化;12 更稳,3h 窗口充裕)
STARTUP_STAGGER = 0.3 # 首批城市启动错峰间隔秒(削平瞬时峰值,实测能压低 402)
PRUNE_FAIL_RATIO_MAX = 0.05 # 失败城占比超此值则本轮跳过 prune(避免大面积抓取失败误删库)
SOURCES = [ SOURCES = [
{"code": "search_waimai", "label": "外卖·搜外卖", "kind": "search", "platform": 1, "keyword": "外卖"}, {"code": "search_waimai", "label": "外卖·搜外卖", "kind": "search", "platform": 1, "keyword": "外卖"},
@@ -58,33 +92,44 @@ SOURCES = [
"platform": 2, "biz_lines": [1, 2, 3, 4]}, "platform": 2, "biz_lines": [1, 2, 3, 4]},
] ]
# 跨线程统计(并发抓取下汇总请求量 / 402 / 放弃页数,供观测限流)
_STATS_LOCK = threading.Lock()
_STATS = {"req": 0, "r402": 0, "err": 0}
def _bump(key: str) -> None:
with _STATS_LOCK:
_STATS[key] += 1
# ───────────────────────── 美团调用 ───────────────────────── # ───────────────────────── 美团调用 ─────────────────────────
def _call_retry(body: dict) -> dict | None: def _call_retry(body: dict) -> dict | None:
"""打美团,402/频繁退避重试;其它错误打印并放弃本页。""" """打美团,402/频繁退避重试;其它错误放弃本页。并发下不逐条 print(避免刷屏),计入 _STATS。"""
for a in range(RETRY): for a in range(RETRY):
_bump("req")
try: try:
return _call(QUERY_PATH, body) return _call(QUERY_PATH, body)
except MeituanCpsError as e: except MeituanCpsError as e:
msg = str(e) msg = str(e)
if "402" in msg or "频繁" in msg: if "402" in msg or "频繁" in msg:
_bump("r402")
time.sleep(2.5 * (a + 1)) time.sleep(2.5 * (a + 1))
continue continue
print(f" [warn] meituan: {msg[:80]}") _bump("err")
return None return None
except Exception as e: # noqa: BLE001 except Exception: # noqa: BLE001
print(f" [warn] {type(e).__name__}: {str(e)[:60]}")
time.sleep(2.0 * (a + 1)) time.sleep(2.0 * (a + 1))
_bump("err")
return None return None
def _pull_search(platform: int, keyword: str) -> list[dict]: def _pull_search(city_id: str, platform: int, keyword: str) -> list[dict]:
rows: list[dict] = [] rows: list[dict] = []
sid = None sid = None
pg = 1 pg = 1
while pg <= MAX_PAGES: while pg <= MAX_PAGES:
body = {"platform": platform, "searchText": keyword, "cityId": CITY_BEIJING, "pageSize": PAGE_SIZE} body = {"platform": platform, "searchText": keyword, "cityId": city_id, "pageSize": PAGE_SIZE}
if sid: if sid:
body["searchId"] = sid body["searchId"] = sid
else: else:
@@ -102,14 +147,14 @@ def _pull_search(platform: int, keyword: str) -> list[dict]:
return rows return rows
def _pull_supply(platform: int, biz_lines: list[int]) -> list[dict]: def _pull_supply(city_id: str, platform: int, biz_lines: list[int]) -> list[dict]:
rows: list[dict] = [] rows: list[dict] = []
sid = None sid = None
biz_param = [{"bizLine": b} for b in biz_lines] biz_param = [{"bizLine": b} for b in biz_lines]
for _ in range(MAX_PAGES): for _ in range(MAX_PAGES):
body = { body = {
"multipleSupplyList": [{"platform": platform, "bizLineParamList": biz_param}], "multipleSupplyList": [{"platform": platform, "bizLineParamList": biz_param}],
"cityId": CITY_BEIJING, "cityId": city_id,
"sortField": 2, # 供给查询 sortField 必填;我们入库后本地再排,这里给个默认 "sortField": 2, # 供给查询 sortField 必填;我们入库后本地再排,这里给个默认
"pageSize": PAGE_SIZE, "pageSize": PAGE_SIZE,
} }
@@ -157,7 +202,19 @@ def _to_cents(yuan) -> int | None:
return None return None
def _parse_item(item: dict, source: dict) -> dict | None: def _strip_nul(v):
"""递归去掉字符串里的 NUL(0x00):PostgreSQL 的 text / jsonb 字段都不接受该字节,
美团偶有脏数据(某券文本含 NUL)会让整批 upsert DataError入库前统一清洗"""
if isinstance(v, str):
return v.replace(chr(0), "")
if isinstance(v, dict):
return {k: _strip_nul(x) for k, x in v.items()}
if isinstance(v, list):
return [_strip_nul(x) for x in v]
return v
def _parse_item(item: dict, source: dict, city_id: str) -> dict | None:
cpd = item.get("couponPackDetail") or {} cpd = item.get("couponPackDetail") or {}
br = item.get("brandInfo") or {} br = item.get("brandInfo") or {}
ci = item.get("commissionInfo") or {} ci = item.get("commissionInfo") or {}
@@ -186,11 +243,12 @@ def _parse_item(item: dict, source: dict) -> dict | None:
dist = None dist = None
dedup_raw = f"{brand}|{name}|{price_cents}" dedup_raw = f"{brand}|{name}|{price_cents}"
return { # 入库前递归清洗 NUL(美团脏数据偶含 0x00,PostgreSQL text/jsonb 拒绝整批 → DataError)
return _strip_nul({
"source": source["code"], "source": source["code"],
"platform": source["platform"], "platform": source["platform"],
"biz_line": item.get("bizLine") or cpd.get("bizLine"), "biz_line": item.get("bizLine") or cpd.get("bizLine"),
"city_id": CITY_BEIJING, "city_id": city_id,
"product_view_sign": str(sign)[:128], "product_view_sign": str(sign)[:128],
"sku_view_id": cpd.get("skuViewId"), "sku_view_id": cpd.get("skuViewId"),
"name": (name[:256] or None), "name": (name[:256] or None),
@@ -207,7 +265,28 @@ def _parse_item(item: dict, source: dict) -> dict | None:
"delivery_distance_m": dist, "delivery_distance_m": dist,
"dedup_key": hashlib.md5(dedup_raw.encode("utf-8")).hexdigest(), "dedup_key": hashlib.md5(dedup_raw.encode("utf-8")).hexdigest(),
"raw": item, "raw": item,
} })
def _pull_one_city(city: dict, index: int, concurrency: int, stagger: float) -> list[dict]:
"""抓单个城市的 3 路券并解析。worker 线程内执行(只抓取+解析,不碰 DB)。
启动错峰:首批并发的 `concurrency` 个城市按 index 错开首请求,削平瞬时峰值(压低 402)
"""
if stagger:
time.sleep((index % concurrency) * stagger)
cid = city["city_id"]
parsed: list[dict] = []
for src in SOURCES:
if src["kind"] == "search":
items = _pull_search(cid, src["platform"], src["keyword"])
else:
items = _pull_supply(cid, src["platform"], src["biz_lines"])
for it in items:
p = _parse_item(it, src, cid)
if p:
parsed.append(p)
return parsed
# ───────────────────────── 入库(upsert) ───────────────────────── # ───────────────────────── 入库(upsert) ─────────────────────────
@@ -272,30 +351,59 @@ def _release_lock() -> None:
# ───────────────────────── 主流程 ───────────────────────── # ───────────────────────── 主流程 ─────────────────────────
def run_once(prune_hours: int = 24) -> None: def run_once(
prune_hours: int = 24,
concurrency: int = DEFAULT_CONCURRENCY,
stagger: float = STARTUP_STAGGER,
cities: list[dict] | None = None,
) -> None:
if not _acquire_lock(): if not _acquire_lock():
print(f"[{datetime.now():%H:%M:%S}] 上一轮还在跑(锁占用),跳过本轮") print(f"[{datetime.now():%H:%M:%S}] 上一轮还在跑(锁占用),跳过本轮")
return return
t0 = time.time() t0 = time.time()
now = datetime.now(timezone.utc) now = datetime.now(timezone.utc)
with _STATS_LOCK:
_STATS.update(req=0, r402=0, err=0)
cities = cities if cities is not None else all_cities()
n_city = len(cities)
db = SessionLocal() db = SessionLocal()
try: try:
total = 0 total_up = 0
for src in SOURCES: done = 0
ts = time.time() fails: list[str] = []
if src["kind"] == "search": # 城市级并发抓取(worker 只抓取+解析),主线程逐城串行入库(Session 不跨线程)。
items = _pull_search(src["platform"], src["keyword"]) with ThreadPoolExecutor(max_workers=concurrency) as pool:
else: futs = {
items = _pull_supply(src["platform"], src["biz_lines"]) pool.submit(_pull_one_city, city, i, concurrency, stagger): city
parsed = [p for p in (_parse_item(it, src) for it in items) if p] for i, city in enumerate(cities)
up, dup = _upsert(db, parsed, now) }
total += up for fut in as_completed(futs):
print(f" {src['label']:18}{len(items):5} 解析{len(parsed):5} " city = futs[fut]
f"入库{up:5} (本源去重{dup:3}) {time.time() - ts:4.0f}s") try:
# 清理长期未再出现的陈旧券(美团 sign 轮换 / 券下架后的残留),默认 24h 宽限。 parsed = fut.result()
# 护栏:仅在本轮确有入库(total>0)时才清理。美团整体故障时本轮可能 0 入库 except Exception as e: # noqa: BLE001
# (脚本不抛异常、只是抓回空),若仍照常 prune,连续故障会按 last_seen 把全表删空。 fails.append(city["name"])
if prune_hours and prune_hours > 0 and total > 0: print(f" [fail] {city['name']}: {type(e).__name__}: {str(e)[:60]}")
continue
try:
up, _dup = _upsert(db, parsed, now)
except Exception as e: # noqa: BLE001
db.rollback() # 事务已 abort,必须 rollback 才能继续给下一城复用 Session
fails.append(city["name"])
print(f" [入库失败] {city['name']}: {type(e).__name__}: {str(e)[:80]}")
continue
total_up += up
done += 1
if done % 20 == 0 or done == n_city:
print(f" [{done}/{n_city}] {city['name']:10}{len(parsed):5} 入库{up:5} "
f"(累计入库 {total_up}, 用时 {time.time() - t0:.0f}s)")
# 清理陈旧券(美团 sign 轮换 / 券下架残留),默认 24h 宽限。两道护栏防误删:
# ① total_up>0:本轮 0 入库(疑似上游整体故障,脚本不抛异常只抓回空)时跳过,否则
# 会按 last_seen 把全表删空;
# ② 失败城占比 ≤5%:大面积城市抓取失败(限流/网络)时跳过,避免误删还在架上的券。
fail_ratio = len(fails) / max(1, n_city)
if prune_hours and prune_hours > 0 and total_up > 0 and fail_ratio <= PRUNE_FAIL_RATIO_MAX:
cutoff = now - timedelta(hours=prune_hours) cutoff = now - timedelta(hours=prune_hours)
pruned = db.execute( pruned = db.execute(
delete(MeituanCoupon).where(MeituanCoupon.last_seen < cutoff) delete(MeituanCoupon).where(MeituanCoupon.last_seen < cutoff)
@@ -303,36 +411,62 @@ def run_once(prune_hours: int = 24) -> None:
db.commit() db.commit()
if pruned: if pruned:
print(f" 清理陈旧券(>{prune_hours}h 未再出现): {pruned}") print(f" 清理陈旧券(>{prune_hours}h 未再出现): {pruned}")
elif prune_hours and prune_hours > 0 and total == 0: elif prune_hours and prune_hours > 0:
print(" 本轮 0 入库(疑似上游故障),跳过清理以防误删全表") reason = "本轮 0 入库" if total_up == 0 else f"失败城占比 {fail_ratio:.0%}>{PRUNE_FAIL_RATIO_MAX:.0%}"
print(f" 跳过 prune({reason},防误删)")
cnt = db.execute(select(func.count()).select_from(MeituanCoupon)).scalar() cnt = db.execute(select(func.count()).select_from(MeituanCoupon)).scalar()
print(f"[{datetime.now():%H:%M:%S}] 本轮完成: 入库 {total} 条, 表总计 {cnt} 行, " with _STATS_LOCK:
f"用时 {time.time() - t0:.0f}s") req, r402, err = _STATS["req"], _STATS["r402"], _STATS["err"]
fail_tail = (": " + ",".join(fails[:10])) if fails else ""
print(f"[{datetime.now():%H:%M:%S}] 本轮完成: {done}/{n_city} 城, 入库 {total_up} 条, "
f"表总计 {cnt} 行, 请求 {req}(402 {r402} / 放弃 {err}), "
f"失败城 {len(fails)}{fail_tail}, 用时 {time.time() - t0:.0f}s")
finally: finally:
db.close() db.close()
_release_lock() _release_lock()
def _select_cities(limit: int, city_ids: str) -> list[dict]:
cities = all_cities()
if city_ids.strip():
want = {c.strip() for c in city_ids.split(",") if c.strip()}
return [c for c in cities if c["city_id"] in want]
if limit and limit > 0:
return cities[:limit]
return cities
def main() -> None: def main() -> None:
ap = argparse.ArgumentParser(description="美团 CPS 券定时抓取入库") ap = argparse.ArgumentParser(description="美团 CPS 券定时抓取入库(全国 359 地级市)")
ap.add_argument("--once", action="store_true", help="只跑一轮(默认)") ap.add_argument("--once", action="store_true", help="只跑一轮(默认)")
ap.add_argument("--loop", action="store_true", help="循环跑") ap.add_argument("--loop", action="store_true", help="循环跑")
ap.add_argument("--interval", type=int, default=600, help="循环间隔秒(默认 600=10min)") ap.add_argument("--interval", type=int, default=10800,
help="循环间隔秒(默认 10800=3h,对齐每 3h 全量一轮)")
ap.add_argument("--prune-hours", type=int, default=24, ap.add_argument("--prune-hours", type=int, default=24,
help="清理超过 N 小时未再出现的陈旧券(默认 24;0=不清理)") help="清理超过 N 小时未再出现的陈旧券(默认 24;0=不清理)")
ap.add_argument("--concurrency", type=int, default=DEFAULT_CONCURRENCY,
help=f"并发城市数(默认 {DEFAULT_CONCURRENCY};实测 15 并发 402 占 3%% 可退避消化)")
ap.add_argument("--stagger", type=float, default=STARTUP_STAGGER,
help=f"首批城市启动错峰间隔秒(默认 {STARTUP_STAGGER};削平瞬时峰值压低 402)")
ap.add_argument("--limit", type=int, default=0, help="只抓前 N 个城市(测试用,0=全部)")
ap.add_argument("--city-ids", default="", help="只抓这些 cityId(逗号分隔,测试用,优先于 --limit)")
args = ap.parse_args() args = ap.parse_args()
cities = _select_cities(args.limit, args.city_ids)
print(f"城市数: {len(cities)} / 并发: {args.concurrency} / 错峰: {args.stagger}s / 间隔: {args.interval}s")
if args.loop: if args.loop:
print(f"循环模式: 每 {args.interval}s 一轮 (Ctrl-C 退出)") print(f"循环模式: 每 {args.interval}s 一轮 (Ctrl-C 退出)")
while True: while True:
try: try:
run_once(args.prune_hours) run_once(args.prune_hours, args.concurrency, args.stagger, cities)
except Exception as e: # noqa: BLE001 except Exception as e: # noqa: BLE001
print(f"[{datetime.now():%H:%M:%S}] 本轮异常: {type(e).__name__}: {e}") print(f"[{datetime.now():%H:%M:%S}] 本轮异常: {type(e).__name__}: {e}")
_release_lock() _release_lock()
time.sleep(args.interval) time.sleep(args.interval)
else: else:
run_once(args.prune_hours) run_once(args.prune_hours, args.concurrency, args.stagger, cities)
if __name__ == "__main__": if __name__ == "__main__":
+34
View File
@@ -117,6 +117,40 @@ def test_update_ad_limit_takes_effect(admin_client: TestClient, token: str) -> N
db.close() db.close()
def test_update_bool_config(admin_client: TestClient, token: str) -> None:
# 提现自动对账开关默认 True
items = {
i["key"]: i
for i in admin_client.get("/admin/api/config", headers=_auth(token)).json()
}
assert items["withdraw_auto_reconcile_enabled"]["type"] == "bool"
assert items["withdraw_auto_reconcile_enabled"]["value"] is True
# 关掉 → DB 落 False、业务读到 False
r = admin_client.patch(
"/admin/api/config/withdraw_auto_reconcile_enabled",
json={"value": False},
headers=_auth(token),
)
assert r.status_code == 200, r.text
assert r.json()["value"] is False and r.json()["overridden"] is True
db = SessionLocal()
try:
from app.repositories import app_config
assert app_config.get_value(db, "withdraw_auto_reconcile_enabled") is False
finally:
db.close()
# bool 项不接受非布尔值
assert admin_client.patch(
"/admin/api/config/withdraw_auto_reconcile_enabled",
json={"value": 1},
headers=_auth(token),
).status_code == 400
def test_config_validation(admin_client: TestClient, token: str) -> None: def test_config_validation(admin_client: TestClient, token: str) -> None:
# 签到档位长度≠7 # 签到档位长度≠7
assert admin_client.patch( assert admin_client.patch(
+37
View File
@@ -206,3 +206,40 @@ def test_sms_gc_purges_stale_only(monkeypatch) -> None:
assert "stale" not in sms._codes and "fresh" in sms._codes assert "stale" not in sms._codes and "fresh" in sms._codes
assert "old" not in sms._last_sent and "recent" in sms._last_sent assert "old" not in sms._last_sent and "recent" in sms._last_sent
assert "yesterday" not in sms._daily_count and "today" in sms._daily_count assert "yesterday" not in sms._daily_count and "today" in sms._daily_count
# ============================ 用户名 / 默认昵称 ============================
def test_login_assigns_username_and_nickname(client) -> None:
"""新用户创建即分配:11 位纯数字 username(首位非 0/1,与手机号天然区分)+ 9 位字母数字默认昵称。"""
phone = "13600136000"
client.post("/api/v1/auth/sms/send", json={"phone": phone})
r = client.post("/api/v1/auth/sms/login", json={"phone": phone, "code": "123456"})
assert r.status_code == 200, r.text
user = r.json()["user"]
uname = user["username"]
assert uname.isdigit() and len(uname) == 11 # 11 位纯数字
assert uname[0] not in ("0", "1") # 无前导 0、与手机号(均以 1 开头)区分
nick = user["nickname"]
assert nick and len(nick) == 9 and nick.isalnum() # 9 位字母+数字
def test_username_stable_and_unique_on_relogin() -> None:
"""同号重登 username 不变(不可变标识);不同号 username 不同(唯一)。
直连 repository 绕开 HTTP 短信冷却"""
from app.db.session import SessionLocal
from app.repositories import user as user_repo
db = SessionLocal()
try:
a1 = user_repo.upsert_user_for_login(db, phone="13522220001", register_channel="sms")
uname_a, id_a = a1.username, a1.id
a2 = user_repo.upsert_user_for_login(db, phone="13522220001", register_channel="sms")
assert a2.id == id_a and a2.username == uname_a # 重登:同一行、username 稳定不变
b = user_repo.upsert_user_for_login(db, phone="13522220002", register_channel="sms")
assert b.username != uname_a # 不同用户、username 唯一
finally:
db.close()
+13 -5
View File
@@ -300,7 +300,7 @@ def test_bind_no_code_no_fingerprint(client) -> None:
# ===================================================================== # =====================================================================
def test_invitees_basic(client) -> None: def test_invitees_basic(client) -> None:
"""A 邀 B、C → 列表返 2 条、total=2、没设资料的名字=脱敏手机号、头像 null、金币对。""" """A 邀 B、C → 列表返 2 条、total=2、名字=被邀请人默认昵称(创建即有)、头像 null、金币对。"""
a = _login(client, "13800002050") a = _login(client, "13800002050")
a_code = _my_code(client, a) a_code = _my_code(client, a)
for phone in ("13800002051", "13800002052"): for phone in ("13800002051", "13800002052"):
@@ -313,9 +313,14 @@ def test_invitees_basic(client) -> None:
assert body["total"] == 2 assert body["total"] == 2
assert body["has_more"] is False assert body["has_more"] is False
assert len(body["items"]) == 2 assert len(body["items"]) == 2
# 没设昵称头像 → 名字脱敏手机号、头像 null(不依赖顺序用 set) # 创建即分配默认昵称 → 名字=被邀请人昵称(取实际值对比)、头像 null(不依赖顺序用 set)
with SessionLocal() as db:
expected_names = {
get_user_by_phone(db, "13800002051").nickname,
get_user_by_phone(db, "13800002052").nickname,
}
names = {it["display_name"] for it in body["items"]} names = {it["display_name"] for it in body["items"]}
assert names == {"138****2051", "138****2052"} assert names == expected_names
assert all(it["avatar_url"] is None for it in body["items"]) assert all(it["avatar_url"] is None for it in body["items"])
assert all(it["coins"] == INVITE_INVITER_COINS for it in body["items"]) assert all(it["coins"] == INVITE_INVITER_COINS for it in body["items"])
@@ -336,9 +341,12 @@ def test_invitees_order_desc(client) -> None:
).scalar_one() ).scalar_one()
rel_b.created_at = datetime.now(timezone.utc) - timedelta(hours=2) rel_b.created_at = datetime.now(timezone.utc) - timedelta(hours=2)
db.commit() db.commit()
nick_b = ub.nickname
nick_c = get_user_by_phone(db, "13800002062").nickname
items = client.get("/api/v1/invite/invitees", headers=_auth(a)).json()["items"] items = client.get("/api/v1/invite/invitees", headers=_auth(a)).json()["items"]
assert items[0]["display_name"] == "138****2062" # C 刚邀,在前 # 创建即有默认昵称 → display_name=被邀请人昵称;本用例核心验证倒序(C 刚邀在前、B 2h 前在后)
assert items[1]["display_name"] == "138****2061" # B 2 小时前,在后 assert items[0]["display_name"] == nick_c
assert items[1]["display_name"] == nick_b
def test_invitees_nickname_priority(client) -> None: def test_invitees_nickname_priority(client) -> None:
+13
View File
@@ -0,0 +1,13 @@
"""平台公开端点测试(不鉴权):/api/v1/platform/flags 等。"""
from __future__ import annotations
from fastapi.testclient import TestClient
def test_flags_default_comparing_ad_enabled(client: TestClient) -> None:
"""空配置库下,/flags 返回 comparing_ad_enabled 的默认值 True;不需要鉴权。"""
r = client.get("/api/v1/platform/flags")
assert r.status_code == 200, r.text
body = r.json()
assert "comparing_ad_enabled" in body
assert body["comparing_ad_enabled"] is True
+156
View File
@@ -0,0 +1,156 @@
"""淘宝 deeplink 失效标记 + lookup 过滤 + invalidate 端点测试。
覆盖: shopId 标记所有行(幂等)lookup 过滤失效淘宝候选失效后新行仍可命中
invalidate 端点鉴权(503 未配 / 401 头错 / 200)端到端标记后 lookup MISS非淘宝 no-op
"""
from __future__ import annotations
import pytest
from app.core.config import settings
from app.db.session import SessionLocal
from app.models.store_mapping import StoreMapping
from app.repositories import store_mapping as repo
_SECRET = "test-internal-secret-only-for-pytest"
def _mk_row(db, *, trace_id, name_meituan=None, name_taobao=None,
id_taobao=None, deeplink=None, lat=None, lng=None,
name_jd=None, id_jd=None, jd_deeplink=None):
row = StoreMapping(
trace_id=trace_id, business_type="food",
name_meituan=name_meituan, name_taobao=name_taobao,
id_taobao=id_taobao, taobao_deeplink=deeplink, lat=lat, lng=lng,
name_jd=name_jd, id_jd=id_jd, jd_deeplink=jd_deeplink,
)
db.add(row)
db.commit()
db.refresh(row)
return row
@pytest.fixture()
def db():
"""每个用例前后清空 store_mapping(表由 conftest 的 create_all 建好,session 级共享)。"""
s = SessionLocal()
s.query(StoreMapping).delete()
s.commit()
yield s
s.query(StoreMapping).delete()
s.commit()
s.close()
# ---------- repo 层 ----------
def test_mark_invalid_marks_all_rows_with_shop_id(db):
# 同一个坏 shopId 散在两行(两次比价),另一行不同 shopId
_mk_row(db, trace_id="t1", name_meituan="绝味鸭脖", name_taobao="绝味鸭脖", id_taobao="BAD1", deeplink="dl_a")
_mk_row(db, trace_id="t2", name_meituan="绝味鸭脖", name_taobao="绝味鸭脖", id_taobao="BAD1", deeplink="dl_b")
_mk_row(db, trace_id="t3", name_meituan="绝味鸭脖", name_taobao="绝味鸭脖", id_taobao="GOOD9", deeplink="dl_g")
# 按 shopId 标记所有行
assert repo.mark_taobao_deeplink_invalid(db, "BAD1") == 2
# 幂等:已标记的不重复
assert repo.mark_taobao_deeplink_invalid(db, "BAD1") == 0
rows = {r.trace_id: r for r in db.query(StoreMapping).all()}
assert rows["t1"].taobao_deeplink_invalid_at is not None
assert rows["t2"].taobao_deeplink_invalid_at is not None
assert rows["t3"].taobao_deeplink_invalid_at is None # 不同 shopId 不动
def test_lookup_filters_invalid_taobao(db):
_mk_row(db, trace_id="t1", name_meituan="绝味鸭脖", name_taobao="绝味鸭脖", id_taobao="BAD1", deeplink="dl")
# 标记前命中
assert repo.lookup_nearest(db, "meituan", "绝味鸭脖")["taobao"]["shop_id"] == "BAD1"
# 标记失效后淘宝候选被过滤 → MISS(仅此一条淘宝)
repo.mark_taobao_deeplink_invalid(db, "BAD1")
assert "taobao" not in repo.lookup_nearest(db, "meituan", "绝味鸭脖")
def test_lookup_picks_new_valid_row_after_invalidate(db):
# 失效旧行 + 重搜写的新行(新 shopId,invalid_at=NULL)共存 → lookup 选到新行
_mk_row(db, trace_id="t1", name_meituan="店A", name_taobao="店A", id_taobao="BAD1", deeplink="dl_bad")
repo.mark_taobao_deeplink_invalid(db, "BAD1")
_mk_row(db, trace_id="t2", name_meituan="店A", name_taobao="店A", id_taobao="NEW2", deeplink="dl_new")
assert repo.lookup_nearest(db, "meituan", "店A")["taobao"]["shop_id"] == "NEW2"
# ---------- invalidate 端点 ----------
def test_invalidate_endpoint_secret_unset_503(client, monkeypatch):
monkeypatch.setattr(settings, "INTERNAL_API_SECRET", "") # 未配 = 端点关闭
r = client.post("/internal/store-mapping/invalidate",
json={"platform": "taobao", "shop_id": "X"})
assert r.status_code == 503
def test_invalidate_endpoint_auth(client, monkeypatch):
monkeypatch.setattr(settings, "INTERNAL_API_SECRET", _SECRET)
body = {"platform": "taobao", "shop_id": "X"}
assert client.post("/internal/store-mapping/invalidate", json=body).status_code == 401
assert client.post("/internal/store-mapping/invalidate", json=body,
headers={"X-Internal-Secret": "wrong"}).status_code == 401
r = client.post("/internal/store-mapping/invalidate", json=body,
headers={"X-Internal-Secret": _SECRET})
assert r.status_code == 200 and r.json()["ok"] is True
def test_invalidate_endpoint_marks_and_lookup_miss(client, db, monkeypatch):
monkeypatch.setattr(settings, "INTERNAL_API_SECRET", _SECRET)
_mk_row(db, trace_id="t1", name_meituan="店B", name_taobao="店B", id_taobao="BADX", deeplink="dl")
_mk_row(db, trace_id="t2", name_meituan="店B", name_taobao="店B", id_taobao="BADX", deeplink="dl2")
r = client.post("/internal/store-mapping/invalidate",
json={"platform": "taobao", "shop_id": "BADX"},
headers={"X-Internal-Secret": _SECRET})
assert r.status_code == 200 and r.json()["affected"] == 2
lk = client.get("/internal/store-mapping/lookup",
params={"source_platform": "meituan", "name": "店B"},
headers={"X-Internal-Secret": _SECRET})
assert "taobao" not in lk.json()
def test_invalidate_endpoint_unsupported_platform_noop(client, monkeypatch):
# 当前支持 taobao/jd;其它平台(如 meituan)no-op 返 affected=0
monkeypatch.setattr(settings, "INTERNAL_API_SECRET", _SECRET)
r = client.post("/internal/store-mapping/invalidate",
json={"platform": "meituan", "shop_id": "X"},
headers={"X-Internal-Secret": _SECRET})
assert r.status_code == 200 and r.json()["affected"] == 0
# ---------- 京东(对称淘宝)----------
def test_mark_jd_invalid_marks_all_rows_with_store_id(db):
_mk_row(db, trace_id="j1", name_meituan="兰州拉面", name_jd="兰州拉面", id_jd="JBAD", jd_deeplink="dl_a")
_mk_row(db, trace_id="j2", name_meituan="兰州拉面", name_jd="兰州拉面", id_jd="JBAD", jd_deeplink="dl_b")
_mk_row(db, trace_id="j3", name_meituan="兰州拉面", name_jd="兰州拉面", id_jd="JGOOD", jd_deeplink="dl_g")
assert repo.mark_jd_deeplink_invalid(db, "JBAD") == 2
assert repo.mark_jd_deeplink_invalid(db, "JBAD") == 0 # 幂等
rows = {r.trace_id: r for r in db.query(StoreMapping).all()}
assert rows["j1"].jd_deeplink_invalid_at is not None
assert rows["j2"].jd_deeplink_invalid_at is not None
assert rows["j3"].jd_deeplink_invalid_at is None # 不同 storeId 不动
def test_lookup_filters_invalid_jd(db):
_mk_row(db, trace_id="j1", name_meituan="兰州拉面", name_jd="兰州拉面", id_jd="JBAD", jd_deeplink="dl")
assert repo.lookup_nearest(db, "meituan", "兰州拉面")["jd"]["store_id"] == "JBAD"
repo.mark_jd_deeplink_invalid(db, "JBAD")
assert "jd" not in repo.lookup_nearest(db, "meituan", "兰州拉面")
def test_invalidate_endpoint_jd_marks_and_lookup_miss(client, db, monkeypatch):
monkeypatch.setattr(settings, "INTERNAL_API_SECRET", _SECRET)
_mk_row(db, trace_id="j1", name_meituan="店J", name_jd="店J", id_jd="JBADX", jd_deeplink="dl")
_mk_row(db, trace_id="j2", name_meituan="店J", name_jd="店J", id_jd="JBADX", jd_deeplink="dl2")
r = client.post("/internal/store-mapping/invalidate",
json={"platform": "jd", "shop_id": "JBADX"},
headers={"X-Internal-Secret": _SECRET})
assert r.status_code == 200 and r.json()["affected"] == 2
lk = client.get("/internal/store-mapping/lookup",
params={"source_platform": "meituan", "name": "店J"},
headers={"X-Internal-Secret": _SECRET})
assert "jd" not in lk.json()
+57
View File
@@ -0,0 +1,57 @@
# -*- coding: utf-8 -*-
"""从美团「城市字典」Excel 生成随仓库的 JSON(app/integrations/data/meituan_cities.json)。
本地一次性工具:美团每年更新城市字典时,把新 Excel 放进来重跑即可
Excel 不入库不进仓库(二进制 openpyxl);生成的 JSON 随仓库提交,部署到服务器后
ETL(scripts/pull_meituan_coupons.py)直接读它遍历全部城市
字典口径:359 个地级市,经实测一个地级市 cityId 已覆盖其下辖县级市(如徐州邳州/新沂),
故无需区县层级
用法:
python tools/gen_meituan_cities.py ["城市字典xxx.xlsx" 路径]
(不传则用默认 e:\\codes\\城市字典2025 (1).xlsx)
"""
from __future__ import annotations
import json
import os
import sys
import openpyxl
HERE = os.path.dirname(os.path.abspath(__file__))
ROOT = os.path.dirname(HERE)
DEFAULT_EXCEL = r"e:\codes\城市字典2025 (1).xlsx"
OUT = os.path.join(ROOT, "app", "integrations", "data", "meituan_cities.json")
def main() -> None:
excel = sys.argv[1] if len(sys.argv) > 1 else DEFAULT_EXCEL
wb = openpyxl.load_workbook(excel, data_only=True)
ws = wb.worksheets[0]
rows = list(ws.iter_rows(values_only=True))
header = rows[0]
cities = []
seen = set()
for r in rows[1:]:
cid = str(r[0]).strip() if r[0] else ""
name = str(r[1]).strip() if len(r) > 1 and r[1] else ""
prov = str(r[2]).strip() if len(r) > 2 and r[2] else ""
if not cid or cid in seen:
continue
seen.add(cid)
cities.append({"city_id": cid, "name": name, "province": prov})
os.makedirs(os.path.dirname(OUT), exist_ok=True)
with open(OUT, "w", encoding="utf-8") as f:
json.dump(cities, f, ensure_ascii=False, indent=1)
print(f"表头: {header}")
print(f"写出 {len(cities)} 城 → {OUT}")
print("样例:", cities[:3])
if __name__ == "__main__":
main()