diff --git a/.trellis/tasks/08-28-sector-capital-radar/implement.md b/.trellis/tasks/08-28-sector-capital-radar/implement.md index 97cfc76..3a1bb84 100644 --- a/.trellis/tasks/08-28-sector-capital-radar/implement.md +++ b/.trellis/tasks/08-28-sector-capital-radar/implement.md @@ -57,11 +57,13 @@ ## 6. 全量验证与审查 -- [ ] 后端：`uv run ruff format --check .`、`uv run ruff check .`、`uv run pyright`、`uv run pytest`。 -- [ ] 前端：`pnpm format:check`、`pnpm lint`、`pnpm typecheck`、`pnpm test`、`pnpm build`。 -- [ ] 根级：`./dev.sh check`、`./dev.sh test`；验证开发和生产 Compose config。 -- [ ] 使用 `trellis-check` 做全范围规范、PRD、跨层字段、单位、空值、版本声明和凭据泄漏检查，并修复发现项。 -- [ ] 评估是否有经用户批准才应提升到 `.trellis/spec/` 的新知识；未经批准不写 Trellis spec。 +- [x] 后端：`uv run ruff format --check .`、`uv run ruff check .`、`uv run pyright`、`uv run pytest`。 +- [x] 前端：`pnpm format:check`、`pnpm lint`、`pnpm typecheck`、`pnpm test`、`pnpm build`。 +- [x] 根级：`./dev.sh check`、`./dev.sh test`；验证开发和生产 Compose config。 +- [x] 使用 `trellis-check` 做全范围规范、PRD、跨层字段、单位、空值、版本声明和凭据泄漏检查，并修复发现项。 +- [x] 评估是否有经用户批准才应提升到 `.trellis/spec/` 的新知识；未经批准不写 Trellis spec。 + +阶段结果（2026-08-29）：全范围终审补齐三项契约：`SourceSnapshot` identity 绑定返回字段、行上限和截断状态；显式空 `dc_member` 分区持久化为 `membership_unknown`，生成 unavailable 聚合并强制 publication 为 partial，只重试成员来源且绝不替换 last-good；HTTP 与前端把排名百分位统一收紧为 `(0, 100]`。新增迁移 head `0006_membership_unknown`，离线升级 SQL 已核对。最终后端 Ruff、Pyright 和全量测试为 139 passed、3 skipped，跳过项均需要 `ZHIXING_TEST_DATABASE_URL`；前端 format、lint、typecheck、全量 64 项 Vitest 与 build 通过，build 仅有既有单包大于 500 kB 的非阻塞提示；`./dev.sh check`、`./dev.sh test` 和开发/生产、默认/jobs 四种 Compose `config --quiet` 均通过。Compose 验证显式清空 `ZHIXING_TUSHARE_TOKEN` 并使用无敏感信息的占位数据库 URL。真实 PostgreSQL 集成、真实 Tushare capability、生产网络和部署权限仍未在本机环境验证，不将其误报为通过。全范围只读复核最终为 no blocking findings。经 `trellis-update-spec` 评估，unknown-membership 与快照 identity 属于可提升的候选知识，但用户未批准写 `.trellis/spec/`，本任务仅在设计、测试和本执行记录中保存。 ## 风险与回滚点 diff --git a/.trellis/tasks/08-28-sector-capital-radar/prd.md b/.trellis/tasks/08-28-sector-capital-radar/prd.md index 87a724f..4450d6c 100644 --- a/.trellis/tasks/08-28-sector-capital-radar/prd.md +++ b/.trellis/tasks/08-28-sector-capital-radar/prd.md @@ -55,14 +55,14 @@ Tushare token 只从 `Settings`/环境注入，不得写入源码、日志、响 ## Acceptance Criteria -- [ ] AC1：给定固定样本和乱序输入，概念/行业的三套排名、排名百分位、普通前后榜和 1—5 日排名变化结果可重复，且测试覆盖空池、单元素池、并列值、历史缺失和非有限数。 -- [ ] AC2：每条雷达结果可追溯到唯一发布版本、来源版本、universe 版本和指标策略版本；响应和页面明确标注“知行独立实现”，不暴露或暗示原站 `Ratio_Score`、`Swing_Score` 字段。 -- [ ] AC3：缺失资金流、成员未知、低流动性、部分覆盖与失败发布不会被展示成完整的零值结果；失败构建不覆盖 `last_good`。 -- [ ] AC4：后端 HTTP 契约测试锁定筛选、分页/榜单、数据状态和错误行为；前端类型、API adapter、query 与页面测试覆盖 loading/error/no-data/stale/partial/success。 -- [ ] AC5：页面可分别浏览概念与行业排名池，并按交易日、指标视角和强弱榜筛选；金额、比例、策略版本与排名变化的单位和方向符合本任务契约。 -- [ ] AC6：收盘后 Job 可幂等重复执行，重复内容不产生无意义修订；同一日期并发执行被锁阻止，失败时保留最近有效发布。 -- [ ] AC7：运行相关后端 Ruff、Pyright、pytest 与前端 format、lint、typecheck、Vitest、build；跨层链路通过根级检查。没有实际运行的检查不得标记为通过。 -- [ ] AC8：生产运行时不请求 OneChartLab；Tushare 原始响应、point-in-time 成员和规范化股票事实足以重放同一指标策略版本，且公开样本只用于对账，不覆盖本地事实。 +- [x] AC1：给定固定样本和乱序输入，概念/行业的三套排名、排名百分位、普通前后榜和 1—5 日排名变化结果可重复，且测试覆盖空池、单元素池、并列值、历史缺失和非有限数。 +- [x] AC2：每条雷达结果可追溯到唯一发布版本、来源版本、universe 版本和指标策略版本；响应和页面明确标注“知行独立实现”，不暴露或暗示原站 `Ratio_Score`、`Swing_Score` 字段。 +- [x] AC3：缺失资金流、成员未知、低流动性、部分覆盖与失败发布不会被展示成完整的零值结果；失败构建不覆盖 `last_good`。 +- [x] AC4：后端 HTTP 契约测试锁定筛选、分页/榜单、数据状态和错误行为；前端类型、API adapter、query 与页面测试覆盖 loading/error/no-data/stale/partial/success。 +- [x] AC5：页面可分别浏览概念与行业排名池，并按交易日、指标视角和强弱榜筛选；金额、比例、策略版本与排名变化的单位和方向符合本任务契约。 +- [x] AC6：收盘后 Job 可幂等重复执行，重复内容不产生无意义修订；同一日期并发执行被锁阻止，失败时保留最近有效发布。 +- [x] AC7：运行相关后端 Ruff、Pyright、pytest 与前端 format、lint、typecheck、Vitest、build；跨层链路通过根级检查。没有实际运行的检查不得标记为通过。 +- [x] AC8：生产运行时不请求 OneChartLab；Tushare 原始响应、point-in-time 成员和规范化股票事实足以重放同一指标策略版本，且公开样本只用于对账，不覆盖本地事实。 ## Out of Scope diff --git a/zhixing-server/migrations/versions/0006_membership_unknown.py b/zhixing-server/migrations/versions/0006_membership_unknown.py new file mode 100644 index 0000000..ab3e2ce --- /dev/null +++ b/zhixing-server/migrations/versions/0006_membership_unknown.py @@ -0,0 +1,103 @@ +"""Persist explicit unknown point-in-time sector membership snapshots.""" + +from collections.abc import Sequence + +import sqlalchemy as sa +from alembic import op + +revision: str = "0006_membership_unknown" +down_revision: str | None = "0005_radar_daily_aggregate" +branch_labels: str | Sequence[str] | None = None +depends_on: str | Sequence[str] | None = None + + +def upgrade() -> None: + """Allow one null-stock marker for an explicitly empty sector partition.""" + + op.add_column( + "sector_radar_membership", + sa.Column("membership_key", sa.String(32), nullable=True), + ) + op.execute("UPDATE sector_radar_membership SET membership_key = stock_code") + op.drop_constraint( + "sector_radar_membership_pkey", + "sector_radar_membership", + type_="primary", + ) + op.drop_constraint( + "ck_sector_radar_membership_status", + "sector_radar_membership", + type_="check", + ) + op.alter_column( + "sector_radar_membership", + "membership_key", + existing_type=sa.String(32), + nullable=False, + ) + op.alter_column( + "sector_radar_membership", + "stock_code", + existing_type=sa.String(12), + nullable=True, + ) + op.alter_column( + "sector_radar_membership", + "stock_name", + existing_type=sa.String(128), + nullable=True, + ) + op.create_primary_key( + "sector_radar_membership_pkey", + "sector_radar_membership", + ["source_snapshot_id", "sector_code", "membership_key"], + ) + op.create_check_constraint( + "ck_sector_radar_membership_status", + "sector_radar_membership", + "(membership_status = 'available' " + "AND stock_code IS NOT NULL AND stock_name IS NOT NULL " + "AND membership_key = stock_code) OR " + "(membership_status = 'membership_unknown' " + "AND stock_code IS NULL AND stock_name IS NULL " + "AND membership_key = '__membership_unknown__')", + ) + + +def downgrade() -> None: + """Discard unknown markers and restore the available-member-only schema.""" + + op.execute("DELETE FROM sector_radar_membership WHERE membership_status = 'membership_unknown'") + op.drop_constraint( + "ck_sector_radar_membership_status", + "sector_radar_membership", + type_="check", + ) + op.drop_constraint( + "sector_radar_membership_pkey", + "sector_radar_membership", + type_="primary", + ) + op.alter_column( + "sector_radar_membership", + "stock_code", + existing_type=sa.String(12), + nullable=False, + ) + op.alter_column( + "sector_radar_membership", + "stock_name", + existing_type=sa.String(128), + nullable=False, + ) + op.drop_column("sector_radar_membership", "membership_key") + op.create_primary_key( + "sector_radar_membership_pkey", + "sector_radar_membership", + ["source_snapshot_id", "sector_code", "stock_code"], + ) + op.create_check_constraint( + "ck_sector_radar_membership_status", + "sector_radar_membership", + "membership_status = 'available'", + ) diff --git a/zhixing-server/src/zhixing_server/modules/sector_radar/application/build.py b/zhixing-server/src/zhixing_server/modules/sector_radar/application/build.py index 5f5c406..8c3a5da 100644 --- a/zhixing-server/src/zhixing_server/modules/sector_radar/application/build.py +++ b/zhixing-server/src/zhixing_server/modules/sector_radar/application/build.py @@ -331,9 +331,12 @@ class BuildSectorRadar: aggregates = self._aggregate(collected) rankings = self._rank(target.trade_date, aggregates) coverage = self._coverage(collected.stock_facts) + membership_complete = all( + item.status is MembershipStatus.AVAILABLE for item in collected.memberships + ) terminal = ( PublicationStatus.SUCCESS - if coverage >= self.coverage_threshold + if membership_complete and coverage >= self.coverage_threshold else PublicationStatus.PARTIAL ) finished = RadarPublication( @@ -348,7 +351,13 @@ class BuildSectorRadar: started_at=started_at, finished_at=self.now_fn(), error_summary=( - None if terminal is PublicationStatus.SUCCESS else "coverage_below_threshold" + None + if terminal is PublicationStatus.SUCCESS + else ( + "membership_unknown" + if not membership_complete + else "coverage_below_threshold" + ) ), ) self.repository.finalize_publication( @@ -360,7 +369,10 @@ class BuildSectorRadar: ), rankings=(RankingRecord(publication_id, ranking) for ranking in rankings), retry_source_groups=( - self._retry_source_groups(collected.stock_facts) + self._retry_source_groups( + collected.memberships, + collected.stock_facts, + ) if terminal is PublicationStatus.PARTIAL else () ), @@ -529,7 +541,15 @@ class BuildSectorRadar: ) memberships = normalize_memberships(indices, members) - candidate_codes = tuple(sorted({item.stock_code for item in memberships})) + candidate_codes = tuple( + sorted( + { + item.stock_code + for item in memberships + if item.status is MembershipStatus.AVAILABLE and item.stock_code is not None + } + ) + ) stock_facts = normalize_stock_facts( target_trade_date=target, candidate_codes=candidate_codes, @@ -568,10 +588,18 @@ class BuildSectorRadar: for item in inputs.stock_facts ) grouped: defaultdict[tuple[SectorType, str, str], list[str]] = defaultdict(list) + unknown: set[tuple[SectorType, str, str]] = set() for member in inputs.memberships: - grouped[(member.sector_type, member.sector_code, member.sector_name)].append( - member.stock_code - ) + key = (member.sector_type, member.sector_code, member.sector_name) + if member.status is MembershipStatus.UNKNOWN: + unknown.add(key) + continue + if member.stock_code is None: + raise ValueError("available membership requires a stock code") + grouped[key].append(member.stock_code) + if unknown & set(grouped): + raise ValueError("sector cannot have both available and unknown membership") + sector_keys = set(grouped) | unknown aggregates = tuple( aggregate_sector_snapshot( SectorMembershipSnapshot( @@ -579,15 +607,16 @@ class BuildSectorRadar: sector_type=sector_type, sector_code=sector_code, sector_name=sector_name, - member_codes=tuple(sorted(member_codes)), - status=MembershipStatus.AVAILABLE, + member_codes=tuple(sorted(grouped.get(key, ()))), + status=( + MembershipStatus.UNKNOWN if key in unknown else MembershipStatus.AVAILABLE + ), source_version=self._universe_version(inputs.membership_snapshots), ), facts, ) - for (sector_type, sector_code, sector_name), member_codes in sorted( - grouped.items(), key=lambda item: (str(item[0][0]), item[0][1]) - ) + for key in sorted(sector_keys, key=lambda item: (str(item[0]), item[1])) + for sector_type, sector_code, sector_name in (key,) ) if not aggregates: raise ValueError("sector universe produced no aggregates") @@ -631,9 +660,12 @@ class BuildSectorRadar: @staticmethod def _retry_source_groups( + memberships: Sequence[MembershipRecord], stock_facts: Sequence[StockFactRecord], ) -> tuple[PublicationSourceGroup, ...]: groups: list[PublicationSourceGroup] = [] + if any(item.status is MembershipStatus.UNKNOWN for item in memberships): + groups.append(PublicationSourceGroup.MEMBERS) statuses = {item.status for item in stock_facts} if statuses & { StockFactStatus.MISSING, diff --git a/zhixing-server/src/zhixing_server/modules/sector_radar/domain/normalize.py b/zhixing-server/src/zhixing_server/modules/sector_radar/domain/normalize.py index 4e3abe7..f1dd6c9 100644 --- a/zhixing-server/src/zhixing_server/modules/sector_radar/domain/normalize.py +++ b/zhixing-server/src/zhixing_server/modules/sector_radar/domain/normalize.py @@ -54,27 +54,53 @@ def normalize_memberships( ), None, ) - records: list[MembershipRecord] = [] + members_by_sector: dict[str, list[SectorMemberRow]] = {code: [] for code in index_by_code} for member in members.rows: index = index_by_code.get(member.sector_code) if index is None: raise SourceContractError("dc_member references a sector outside dc_index") - snapshot_id = partition_ids.get(member.sector_code, all_snapshot_id) + members_by_sector[member.sector_code].append(member) + + records: list[MembershipRecord] = [] + for sector_code in sorted(index_by_code): + index = index_by_code[sector_code] + sector_members = members_by_sector[sector_code] + snapshot_id = partition_ids.get(sector_code, all_snapshot_id) if snapshot_id is None: - raise SourceContractError("membership row has no source snapshot") - records.append( - MembershipRecord( - source_snapshot_id=snapshot_id, - trade_date=member.trade_date, - sector_type=index.sector_type, - sector_code=index.sector_code, - sector_name=index.name, - stock_code=member.stock_code, - stock_name=member.stock_name, - status=MembershipStatus.AVAILABLE, + raise SourceContractError("sector membership has no source snapshot") + if not sector_members: + explicit_partition_id = partition_ids.get(sector_code) + if explicit_partition_id is None: + raise SourceContractError( + "missing sector membership requires an explicit empty partition" + ) + records.append( + MembershipRecord( + source_snapshot_id=explicit_partition_id, + trade_date=index.trade_date, + sector_type=index.sector_type, + sector_code=index.sector_code, + sector_name=index.name, + stock_code=None, + stock_name=None, + status=MembershipStatus.UNKNOWN, + ) ) - ) - return tuple(sorted(records, key=lambda item: (item.sector_code, item.stock_code))) + continue + for member in sector_members: + records.append( + MembershipRecord( + source_snapshot_id=snapshot_id, + trade_date=member.trade_date, + sector_type=index.sector_type, + sector_code=index.sector_code, + sector_name=index.name, + stock_code=member.stock_code, + stock_name=member.stock_name, + status=MembershipStatus.AVAILABLE, + ) + ) + return tuple(sorted(records, key=lambda item: (item.sector_code, item.membership_key))) def normalize_stock_facts( diff --git a/zhixing-server/src/zhixing_server/modules/sector_radar/domain/persistence.py b/zhixing-server/src/zhixing_server/modules/sector_radar/domain/persistence.py index 7286206..1c83b82 100644 --- a/zhixing-server/src/zhixing_server/modules/sector_radar/domain/persistence.py +++ b/zhixing-server/src/zhixing_server/modules/sector_radar/domain/persistence.py @@ -33,28 +33,36 @@ def _validate_optional_decimal(value: Decimal | None, field_name: str) -> None: @dataclass(frozen=True, slots=True) class MembershipRecord: - """One persisted point-in-time member tied to its raw source revision.""" + """One persisted point-in-time member or explicit unknown snapshot.""" source_snapshot_id: str trade_date: date sector_type: SectorType sector_code: str sector_name: str - stock_code: str - stock_name: str + stock_code: str | None + stock_name: str | None status: MembershipStatus = MembershipStatus.AVAILABLE def __post_init__(self) -> None: - """Validate revision identity and member fields.""" + """Validate available and unknown membership null semantics.""" _validate_digest(self.source_snapshot_id, "source_snapshot_id") - if self.status is not MembershipStatus.AVAILABLE: - raise ValueError("persisted member rows require available membership") - if any( - not value.strip() - for value in (self.sector_code, self.sector_name, self.stock_code, self.stock_name) - ): - raise ValueError("membership identity fields must not be empty") + if not self.sector_code.strip() or not self.sector_name.strip(): + raise ValueError("membership sector identity fields must not be empty") + if self.status is MembershipStatus.AVAILABLE: + if self.stock_code is None or self.stock_name is None: + raise ValueError("available membership requires stock identity") + if not self.stock_code.strip() or not self.stock_name.strip(): + raise ValueError("available membership stock identity must not be empty") + elif self.stock_code is not None or self.stock_name is not None: + raise ValueError("unknown membership must not expose stock identity") + + @property + def membership_key(self) -> str: + """Return a non-null persistence key without inventing a stock code.""" + + return self.stock_code if self.stock_code is not None else "__membership_unknown__" @dataclass(frozen=True, slots=True) diff --git a/zhixing-server/src/zhixing_server/modules/sector_radar/domain/source.py b/zhixing-server/src/zhixing_server/modules/sector_radar/domain/source.py index a8b8c59..c9f0d5b 100644 --- a/zhixing-server/src/zhixing_server/modules/sector_radar/domain/source.py +++ b/zhixing-server/src/zhixing_server/modules/sector_radar/domain/source.py @@ -122,23 +122,6 @@ def build_source_snapshot( normalized_params = tuple( sorted((key, str(value)) for key, value in params.items() if key != "token") ) - canonical_rows = sorted( - json.dumps(row, ensure_ascii=False, sort_keys=True, separators=(",", ":")) - for row in normalized_rows - ) - content_sha256 = hashlib.sha256("\n".join(canonical_rows).encode()).hexdigest() - identity = json.dumps( - { - "api_name": api_name, - "params": normalized_params, - "partition_key": partition_key, - "content_sha256": content_sha256, - }, - ensure_ascii=False, - sort_keys=True, - separators=(",", ":"), - ) - snapshot_id = hashlib.sha256(identity.encode()).hexdigest() fields = tuple( sorted( set(returned_fields) @@ -147,6 +130,38 @@ def build_source_snapshot( ) ) row_count = len(normalized_rows) + limit_reached = row_limit is not None and row_count >= row_limit + canonical_rows = sorted( + json.dumps(row, ensure_ascii=False, sort_keys=True, separators=(",", ":")) + for row in normalized_rows + ) + canonical_content = json.dumps( + { + "rows": canonical_rows, + "returned_fields": fields, + "row_limit": row_limit, + "limit_reached": limit_reached, + }, + ensure_ascii=False, + sort_keys=True, + separators=(",", ":"), + ) + content_sha256 = hashlib.sha256(canonical_content.encode()).hexdigest() + identity = json.dumps( + { + "api_name": api_name, + "params": normalized_params, + "partition_key": partition_key, + "target_trade_date": ( + target_trade_date.isoformat() if target_trade_date is not None else None + ), + "content_sha256": content_sha256, + }, + ensure_ascii=False, + sort_keys=True, + separators=(",", ":"), + ) + snapshot_id = hashlib.sha256(identity.encode()).hexdigest() return SourceSnapshot( snapshot_id=snapshot_id, api_name=api_name, @@ -159,7 +174,7 @@ def build_source_snapshot( returned_fields=fields, content_sha256=content_sha256, row_limit=row_limit, - limit_reached=row_limit is not None and row_count >= row_limit, + limit_reached=limit_reached, ) diff --git a/zhixing-server/src/zhixing_server/modules/sector_radar/infrastructure/memory.py b/zhixing-server/src/zhixing_server/modules/sector_radar/infrastructure/memory.py index bc97663..903f21e 100644 --- a/zhixing-server/src/zhixing_server/modules/sector_radar/infrastructure/memory.py +++ b/zhixing-server/src/zhixing_server/modules/sector_radar/infrastructure/memory.py @@ -108,7 +108,11 @@ class InMemorySectorRadarRepository: return self._insert_immutable( self.memberships, records, - key=lambda item: (item.source_snapshot_id, item.sector_code, item.stock_code), + key=lambda item: ( + item.source_snapshot_id, + item.sector_code, + item.membership_key, + ), ) def save_stock_facts(self, records: Iterable[StockFactRecord]) -> WriteCounts: diff --git a/zhixing-server/src/zhixing_server/modules/sector_radar/infrastructure/postgres.py b/zhixing-server/src/zhixing_server/modules/sector_radar/infrastructure/postgres.py index 656d6a5..1b5be9e 100644 --- a/zhixing-server/src/zhixing_server/modules/sector_radar/infrastructure/postgres.py +++ b/zhixing-server/src/zhixing_server/modules/sector_radar/infrastructure/postgres.py @@ -246,7 +246,11 @@ class PostgresSectorRadarRepository: items = tuple(records) self._require_unique( items, - lambda item: (item.source_snapshot_id, item.sector_code, item.stock_code), + lambda item: ( + item.source_snapshot_id, + item.sector_code, + item.membership_key, + ), ) rows = tuple( ( @@ -255,6 +259,7 @@ class PostgresSectorRadarRepository: item.sector_type.value, item.sector_code, item.sector_name, + item.membership_key, item.stock_code, item.stock_name, item.status.value, @@ -269,11 +274,12 @@ class PostgresSectorRadarRepository: "sector_type", "sector_code", "sector_name", + "membership_key", "stock_code", "stock_name", "membership_status", ), - ("source_snapshot_id", "sector_code", "stock_code"), + ("source_snapshot_id", "sector_code", "membership_key"), rows, ) @@ -412,7 +418,11 @@ class PostgresSectorRadarRepository: ranking_items = tuple(rankings) self._require_unique( membership_items, - lambda item: (item.source_snapshot_id, item.sector_code, item.stock_code), + lambda item: ( + item.source_snapshot_id, + item.sector_code, + item.membership_key, + ), ) self._require_unique(stock_items, lambda item: (item.fact_revision, item.ts_code)) self._require_unique( @@ -442,11 +452,12 @@ class PostgresSectorRadarRepository: "sector_type", "sector_code", "sector_name", + "membership_key", "stock_code", "stock_name", "membership_status", ), - ("source_snapshot_id", "sector_code", "stock_code"), + ("source_snapshot_id", "sector_code", "membership_key"), tuple( ( item.source_snapshot_id, @@ -454,6 +465,7 @@ class PostgresSectorRadarRepository: item.sector_type.value, item.sector_code, item.sector_name, + item.membership_key, item.stock_code, item.stock_name, item.status.value, diff --git a/zhixing-server/src/zhixing_server/modules/sector_radar/infrastructure/tushare.py b/zhixing-server/src/zhixing_server/modules/sector_radar/infrastructure/tushare.py index 4e48a83..b5ce10d 100644 --- a/zhixing-server/src/zhixing_server/modules/sector_radar/infrastructure/tushare.py +++ b/zhixing-server/src/zhixing_server/modules/sector_radar/infrastructure/tushare.py @@ -229,8 +229,6 @@ class TushareSectorRadarAdapter: self._require_target_date(partition_rows, trade_date, "dc_member") if any(row.sector_code != sector_code for row in partition_rows): raise SourceContractError("dc_member partition returned a different sector") - if not partition_rows: - raise SourceContractError("dc_member cannot prove complete membership for a sector") snapshots.append(snapshot) merged_rows.extend(partition_rows) @@ -240,7 +238,12 @@ class TushareSectorRadarAdapter: api_name="dc_member", ) final_codes = {row.sector_code for row in merged_rows} - if set(expected_codes) - final_codes: + explicitly_observed_codes = { + snapshot.partition_key + for snapshot in snapshots + if snapshot.partition_key not in {None, "all"} + } + if set(expected_codes) - final_codes - explicitly_observed_codes: raise SourceContractError("dc_member response is missing expected sectors") return SourceResult( tuple(snapshots), diff --git a/zhixing-server/src/zhixing_server/modules/sector_radar/presentation/http.py b/zhixing-server/src/zhixing_server/modules/sector_radar/presentation/http.py index e66d4cd..10c9892 100644 --- a/zhixing-server/src/zhixing_server/modules/sector_radar/presentation/http.py +++ b/zhixing-server/src/zhixing_server/modules/sector_radar/presentation/http.py @@ -94,7 +94,7 @@ class RadarRankingRowResponse(BaseModel): membership_coverage: Decimal = Field(ge=0, le=1) moneyflow_coverage: Decimal = Field(ge=0, le=1) rank_position: int | None = Field(default=None, ge=1) - rank_percentile: Decimal | None = Field(default=None, ge=0, le=100) + rank_percentile: Decimal | None = Field(default=None, gt=0, le=100) rank_change_days: int = Field(ge=1, le=5) rank_change: int | None diff --git a/zhixing-server/tests/test_sector_radar_http.py b/zhixing-server/tests/test_sector_radar_http.py index 8eafcdf..2aa7ee9 100644 --- a/zhixing-server/tests/test_sector_radar_http.py +++ b/zhixing-server/tests/test_sector_radar_http.py @@ -2,7 +2,9 @@ from dataclasses import replace from datetime import UTC, date, datetime, timedelta from decimal import Decimal +import pytest from fastapi.testclient import TestClient +from pydantic import ValidationError from zhixing_server.bootstrap.app import create_app from zhixing_server.modules.sector_radar.application.read import ( @@ -29,6 +31,7 @@ from zhixing_server.modules.sector_radar.infrastructure.postgres import ( SectorRadarRepositoryError, ) from zhixing_server.modules.sector_radar.presentation.http import ( + RadarRankingRowResponse, get_sector_radar_reader, ) @@ -191,6 +194,14 @@ def test_no_data_is_a_stable_200_response() -> None: assert rankings.json()["rows"] == [] +def test_http_contract_rejects_zero_rank_percentile() -> None: + payload = _client(FakeReader()).get("/api/v1/sector-radar/rankings").json()["rows"][0] + payload["rank_percentile"] = "0" + + with pytest.raises(ValidationError): + RadarRankingRowResponse.model_validate(payload) + + def test_invalid_query_values_return_422() -> None: client = _client(FakeReader()) diff --git a/zhixing-server/tests/unit/sector_radar/test_build.py b/zhixing-server/tests/unit/sector_radar/test_build.py index 2ebd473..57bff67 100644 --- a/zhixing-server/tests/unit/sector_radar/test_build.py +++ b/zhixing-server/tests/unit/sector_radar/test_build.py @@ -7,6 +7,7 @@ from zhixing_server.modules.sector_radar.application.build import ( BuildSectorRadarCommand, ) from zhixing_server.modules.sector_radar.domain.models import ( + MembershipStatus, PublicationStatus, RadarPublication, SectorType, @@ -19,6 +20,7 @@ from zhixing_server.modules.sector_radar.domain.source import ( SectorIndexRow, SectorMemberRow, SourceResult, + SourceSnapshot, StockBasicRow, SuspendRow, TradeCalendarRow, @@ -37,9 +39,11 @@ class FakeRadarSource: self, *, missing_moneyflow: bool = False, + missing_membership: bool = False, net_scale: Decimal = Decimal(1), ) -> None: self.missing_moneyflow = missing_moneyflow + self.missing_membership = missing_membership self.net_scale = net_scale self.fail_daily = False self.calls: list[str] = [] @@ -153,6 +157,38 @@ class FakeRadarSource: self, trade_date: date, sector_codes: Sequence[str] ) -> SourceResult[SectorMemberRow]: self.calls.append("members") + if self.missing_membership: + snapshots: list[SourceSnapshot] = [] + member_rows: list[SectorMemberRow] = [] + for sector_code in sector_codes: + sector_rows = ( + () + if sector_code == sector_codes[-1] + else tuple( + SectorMemberRow( + trade_date, + sector_code, + f"00000{index}.SZ", + f"股票{index}", + ) + for index in range(1, 6) + ) + ) + snapshots.append( + build_source_snapshot( + api_name="dc_member", + params={ + "trade_date": trade_date.isoformat(), + "ts_code": sector_code, + }, + rows=tuple(self._raw_row(row) for row in sector_rows), + target_trade_date=trade_date, + partition_key=sector_code, + observed_at=NOW, + ) + ) + member_rows.extend(sector_rows) + return SourceResult(tuple(snapshots), tuple(member_rows)) rows = tuple( SectorMemberRow(trade_date, sector_code, f"00000{index}.SZ", f"股票{index}") for sector_code in sector_codes @@ -288,6 +324,33 @@ def test_partial_coverage_and_lock_have_distinct_exit_codes() -> None: assert locked.outcomes[0].status == "locked" +def test_unknown_membership_is_persisted_as_partial_and_retried_independently() -> None: + repository = InMemorySectorRadarRepository() + source = FakeRadarSource(missing_membership=True) + use_case = BuildSectorRadar(source, repository, today=TARGET_DATE, now_fn=lambda: NOW) + + partial = use_case.execute(BuildSectorRadarCommand(trade_date=TARGET_DATE)) + partial_id = partial.outcomes[0].publication_id + assert partial_id is not None + publication = repository.get_publication(partial_id) + assert partial.status == "partial" + assert publication is not None + assert publication.error_summary == "membership_unknown" + assert any(item.status is MembershipStatus.UNKNOWN for item in repository.memberships.values()) + assert { + record.source_group + for record in repository.load_publication_sources(partial_id) + if record.refresh_on_retry + } == {PublicationSourceGroup.MEMBERS} + + source.missing_membership = False + source.calls.clear() + retried = use_case.execute(BuildSectorRadarCommand(retry_publication_id=partial_id)) + + assert retried.status == "success" + assert source.calls == ["members"] + + def test_range_builds_dates_in_order_and_retry_uses_old_target() -> None: repository = InMemorySectorRadarRepository() source = FakeRadarSource(missing_moneyflow=True) diff --git a/zhixing-server/tests/unit/sector_radar/test_normalize.py b/zhixing-server/tests/unit/sector_radar/test_normalize.py index d580249..b639e49 100644 --- a/zhixing-server/tests/unit/sector_radar/test_normalize.py +++ b/zhixing-server/tests/unit/sector_radar/test_normalize.py @@ -1,11 +1,20 @@ from datetime import UTC, date, datetime from decimal import Decimal -from zhixing_server.modules.sector_radar.domain.models import StockFactStatus -from zhixing_server.modules.sector_radar.domain.normalize import normalize_stock_facts +from zhixing_server.modules.sector_radar.domain.models import ( + MembershipStatus, + SectorType, + StockFactStatus, +) +from zhixing_server.modules.sector_radar.domain.normalize import ( + normalize_memberships, + normalize_stock_facts, +) from zhixing_server.modules.sector_radar.domain.source import ( DailyRow, MoneyflowDcRow, + SectorIndexRow, + SectorMemberRow, SourceResult, StockBasicRow, SuspendRow, @@ -64,6 +73,69 @@ def moneyflow(ts_code: str, amount: Decimal | None) -> MoneyflowDcRow: ) +def test_membership_normalization_persists_an_explicit_unknown_sector() -> None: + indices = ( + SectorIndexRow( + TARGET_DATE, + SectorType.CONCEPT, + "BK0001.DC", + "机器人", + "一级", + Decimal(1), + None, + ), + SectorIndexRow( + TARGET_DATE, + SectorType.CONCEPT, + "BK0002.DC", + "低空经济", + "一级", + Decimal(1), + None, + ), + ) + member = SectorMemberRow( + TARGET_DATE, + "BK0001.DC", + "000001.SZ", + "平安银行", + ) + all_snapshot = build_source_snapshot( + api_name="dc_member", + params={"trade_date": "20260828"}, + rows=( + { + "trade_date": "20260828", + "ts_code": member.sector_code, + "con_code": member.stock_code, + "name": member.stock_name, + }, + ), + target_trade_date=TARGET_DATE, + partition_key="all", + observed_at=OBSERVED_AT, + ) + empty_partition = build_source_snapshot( + api_name="dc_member", + params={"trade_date": "20260828", "ts_code": "BK0002.DC"}, + rows=(), + target_trade_date=TARGET_DATE, + partition_key="BK0002.DC", + observed_at=OBSERVED_AT, + ) + + records = normalize_memberships( + indices, + SourceResult((all_snapshot, empty_partition), (member,)), + ) + + assert records[0].status is MembershipStatus.AVAILABLE + assert records[0].stock_code == "000001.SZ" + assert records[1].status is MembershipStatus.UNKNOWN + assert records[1].stock_code is None + assert records[1].membership_key == "__membership_unknown__" + + def test_stock_fact_normalization_preserves_all_missing_and_zero_states() -> None: codes = tuple(f"00000{index}.SZ" for index in range(1, 9)) basics = tuple( diff --git a/zhixing-server/tests/unit/sector_radar/test_tushare_source.py b/zhixing-server/tests/unit/sector_radar/test_tushare_source.py index 7b8fccf..d292467 100644 --- a/zhixing-server/tests/unit/sector_radar/test_tushare_source.py +++ b/zhixing-server/tests/unit/sector_radar/test_tushare_source.py @@ -193,6 +193,37 @@ def test_dc_member_reloads_by_sector_when_the_all_market_call_hits_limit( ] +def test_dc_member_preserves_an_explicit_empty_partition() -> None: + client = QueryClient( + { + ( + "dc_member", + "", + ): ( + { + "trade_date": "20260828", + "ts_code": "BK0001.DC", + "con_code": "000001.SZ", + "name": "A", + }, + ), + ("dc_member", "BK0002.DC"): (), + } + ) + + result = make_adapter(client).fetch_sector_members( + TARGET_DATE, + ("BK0001.DC", "BK0002.DC"), + ) + + assert [row.sector_code for row in result.rows] == ["BK0001.DC"] + assert [snapshot.partition_key for snapshot in result.snapshots] == [ + "all", + "BK0002.DC", + ] + assert result.snapshots[1].row_count == 0 + + def test_stock_basic_explicitly_requests_all_lifecycle_statuses() -> None: responses = { ( @@ -240,6 +271,41 @@ def test_source_snapshot_hash_is_order_stable_and_excludes_token_params() -> Non assert "secret" not in repr(first) +def test_source_snapshot_identity_includes_schema_and_limit_metadata() -> None: + first = build_source_snapshot( + api_name="daily", + params={"trade_date": "20260828"}, + rows=({"ts_code": "000001.SZ"},), + target_trade_date=TARGET_DATE, + observed_at=OBSERVED_AT, + returned_fields=("ts_code",), + row_limit=1, + ) + changed_schema = build_source_snapshot( + api_name="daily", + params={"trade_date": "20260828"}, + rows=({"ts_code": "000001.SZ"},), + target_trade_date=TARGET_DATE, + observed_at=OBSERVED_AT, + returned_fields=("name", "ts_code"), + row_limit=1, + ) + changed_limit = build_source_snapshot( + api_name="daily", + params={"trade_date": "20260828"}, + rows=({"ts_code": "000001.SZ"},), + target_trade_date=TARGET_DATE, + observed_at=OBSERVED_AT, + returned_fields=("ts_code",), + row_limit=2, + ) + + assert first.content_sha256 != changed_schema.content_sha256 + assert first.snapshot_id != changed_schema.snapshot_id + assert first.content_sha256 != changed_limit.content_sha256 + assert first.snapshot_id != changed_limit.snapshot_id + + def test_capability_probe_classifies_errors_without_exposing_provider_text() -> None: client = QueryClient({("daily", ""): RuntimeError("权限不足 private-detail")}) diff --git a/zhixing-web/src/features/sector-radar/api/sector-radar.api.test.ts b/zhixing-web/src/features/sector-radar/api/sector-radar.api.test.ts index 561ddf2..a77a8be 100644 --- a/zhixing-web/src/features/sector-radar/api/sector-radar.api.test.ts +++ b/zhixing-web/src/features/sector-radar/api/sector-radar.api.test.ts @@ -160,4 +160,23 @@ describe("sector radar API adapters", () => { }), ).rejects.toThrow("rankings.rows[0].metric_value must be finite") }) + + it("rejects a zero rank percentile that the domain cannot produce", async () => { + requestJson.mockResolvedValue({ + ...rankingPayload, + rows: [{ ...rankingPayload.rows[0], rank_percentile: "0" }], + }) + + await expect( + getSectorRadarRankings({ + sectorType: "concept", + view: "amount", + rankChangeMetric: "amount", + rankChangeDays: 1, + side: "all", + page: 1, + pageSize: 20, + }), + ).rejects.toThrow("rankings.rows[0].rank_percentile must be greater than 0") + }) }) diff --git a/zhixing-web/src/features/sector-radar/api/sector-radar.api.ts b/zhixing-web/src/features/sector-radar/api/sector-radar.api.ts index 9b9ead4..dd5e1ba 100644 --- a/zhixing-web/src/features/sector-radar/api/sector-radar.api.ts +++ b/zhixing-web/src/features/sector-radar/api/sector-radar.api.ts @@ -270,9 +270,8 @@ function readRankingRow(value: unknown, index: number): RadarRankingRow { 1, `${path}.rank_position`, ), - rank_percentile: readNullableNumberInRange( + rank_percentile: readNullablePositiveNumber( record.rank_percentile, - 0, 100, `${path}.rank_percentile`, ), @@ -403,16 +402,15 @@ function readNullableInteger( return readIntegerInRange(value, min, Number.MAX_SAFE_INTEGER, path) } -function readNullableNumberInRange( +function readNullablePositiveNumber( value: unknown, - min: number, max: number, path: string, ): number | null { if (value === null) return null const number = readFiniteNumber(value, path) - if (number < min || number > max) { - throw contractError(path, `must be between ${min} and ${max}`) + if (number <= 0 || number > max) { + throw contractError(path, `must be greater than 0 and at most ${max}`) } return number }