[Bug] Sampled auto analyze still runs an unsampled MIN/MAX full scan per column #68895

Activity

1. Doris-Breakwater commented on Oct 10, 2026

   

   Doris-Breakwater

   on Oct 10, 2026

   Collaborator

   More actions

   Source review confirms that sampled collection performs a separate, table-wide MIN/MAX query per selected column/index task. This merits statistics-collection scalability investigation. However, an unsampled SQL statement does not by itself establish a full scan of column data: eligible Doris MIN/MAX queries already use segment zone maps. The reported S3 throttling and ingestion impact still need execution evidence.

   Verified against the public 4.1.4 tag (commit ad35a140c7fd0b842f18c23300bac581f7d04326) and master commit dd456ab7a894f54c6553415223c6d5c9c7d4e035:

   1. Sampling does not constrain the preliminary MIN/MAX query. doSample() calls collectMinMax() before choosing sample tablets. BASIC_STATS_TEMPLATE contains neither a tablet/sample restriction nor a limit; only the subsequent statistics query receives TABLET(...) and, where applicable, LIMIT. The preliminary bounds are injected into that subsequent query. An exception from MIN/MAX is rethrown, preventing the subsequent sample query for that task. This applies to columns/indexes selected for collection, rather than necessarily every table column in every scheduler cycle. See 4.1.4 OlapAnalysisTask and sample parameter construction.

   2. Zone-map collection already exists, with eligibility/fallback conditions. The internal statistics context enables MIN/MAX pushdown for unique tables and string columns. Nereids can produce a storage-layer MIN/MAX aggregate; in 4.1.4, BE uses VStatisticsIterator when pushdown is enabled and there are no delete-condition predicates. That iterator obtains two bound values per segment through next_batch_of_zone_map(), instead of reading all column values. Otherwise, BE can use SegmentIterator. See internal context, planner rules, iterator selection, and zone-map iterator. Even the metadata path traverses the table's segments and can incur remote metadata reads on misses; the actual request volume depends on segment count and cache residency.

      Master retains the separate unsampled query, but its internal context now sets forcePushDownZonemapMinMax=true, with corresponding BE handling for truncated bounds and delete predicates. Thus the SQL shape is the same, while fallback behavior is not identical. See master context and master BE handling. This does not remove table-wide traversal.

   3. Cache and timeout settings require the internal-query context. The statistics context disables page cache and sets cloud disableFileCache=true. BE maps that flag to disposable I/O/cache classification; already downloaded file-cache blocks can still be reused. It is not sufficient evidence of complete cache bypass. See BE propagation, cache classification, and cached-block handling. Query timeout is explicitly taken from global analyze_timeout; the 4.1.4 source default is 43,200 seconds. A 300-second failure therefore needs the effective setting and exact error to identify the timeout source. See timeout setting.

   4. [Bug] Failed auto analyze jobs don't record an analyze time, so they restart every cycle #68896 can amplify this cost. Failed terminal jobs skip updateTableStats(), whereas successful jobs update column statistics metadata and lastAnalyzeTime. This supports repeated collection eligibility after failure, but does not establish the observed cadence. Also, in the inspected 4.1.4 production Java source, getHugeTableAutoAnalyzeIntervalInMillis() has no callers; the active collector filters use column eligibility/change checks. The deployed scheduling behavior should be verified rather than relying on that variable as a cooldown. See terminal job handling and collector eligibility.

   To establish the expensive execution path and its effect, please provide:

   • Exact FE/BE build revisions, anonymized SHOW CREATE TABLE (key model, merge-on-write setting, column types, partitions and indexes), and tablet/rowset/segment counts.
   • One affected MIN/MAX query ID, complete SQL, audit entry, full timeout/error message, and matching FE/BE log excerpts. Include effective global analyze_timeout, sample settings, collection window, and analyze-task concurrency.
   • The actual internal-query plan and full profile, including pushed aggregation, rows/data pages read, segment counts, remote/local read bytes and requests, and segment-metadata load time. A manual EXPLAIN must reproduce the internal pushdown settings; ordinary session defaults differ. enable_profile_when_analyze is the existing FE switch for collecting analyze profiles.
   • A timestamp-aligned sample of S3 503/curl errors, request-rate metrics and load latency, plus analyze job/task history across retries. Please redact credentials and sensitive object paths. A controlled off-peak comparison with collection paused for this table would help establish attribution.

   For immediate containment, the existing collection window can move future work off peak. Alternatively, temporarily exclude only this table with ALTER TABLE db.tbl SET ("auto_analyze_policy" = "disable"), restoring its previous policy afterward; this trades statistics freshness for reduced background work and does not cancel already running tasks. These gates are checked by canCollect()/column eligibility and the table policy. Reducing sample rows leaves the preliminary MIN/MAX query unchanged.

   For maintainers, first distinguish missing pushdown/full data reads from expensive table-wide metadata traversal. If the latter dominates, evaluate batching bounds across columns or reusing metadata with correct version visibility, and an explicit option to omit table-wide bounds in sampled collection. Unknown bounds have an existing representation: ColumnStatistic deserialization maps absent bounds to infinities. Sample extrema should not silently become authoritative table-wide bounds, since rare extreme values can be missed and distort selectivity estimates. Any change needs coverage for outliers, nulls, long strings, deletes, key models/indexes and optimizer estimates. Handle retry backoff separately in #68896, preserving the distinction between a failed attempt and successfully refreshed statistics.

   This is source-level triage; no runtime reproduction or supplied profile was available, and no code was changed.

   Breakwater-GitHub-Analysis-Slot: slot_75984b2e62e9
2. Rishavbhattarai commented on Oct 22, 2026

   

   Rishavbhattarai

   on Oct 22, 2026

   More actions

   I'd like to take this one too (see my note on #68896).

   Proposal: in OlapAnalysisTask.doSample(), decide the sample algorithm before collecting min/max.

   • If the algorithm falls back to FULL, the analyze query already scans the whole table. So MIN/MAX can be computed inline in that query instead of in a separate collectMinMax() query. That gives the same result with one query fewer per column.
   • For a real sample (LINEAR/DUJ1), keep the exact collectMinMax() by default, and add a global session variable enable_sample_min_max (default false) that takes min/max from the sampled tablets instead. That trades range accuracy for far fewer reads on huge cloud tables.

   Does that split sound right to you, or would you rather source min/max differently (e.g. from rowset/segment metadata)?

   @englefly @morrySnow