Skip to content

Import files

Some estates cannot hand Venturi a live credential on day one. The provider API is behind a procurement gate, the billing export lands in someone else’s bucket, or the only record of a gateway’s traffic is a log file an engineer can export. File import is the ingestion path specified for exactly that situation: you would upload a provider export or a log file, and Venturi would treat it as a first-class source rather than a workaround.

Target-state: file import is specified, not yet shipped

Everything on this page is the specified contract for file-based ingestion, published so that reviewers can evaluate the design and plan around it. It is not a description of behavior available in the product today. There is no upload surface, no import API, and no registered file drop you can use right now. Nothing here should be read in bare present tense, and no procurement or architecture decision should assume it is live. Release-state labels on this page follow the binding index in Docs authority & product state. The paths that work today are the cloud connectors, the Ingestion API, and the drop-in proxy.

The contract in one sentence

A file-derived event is specified to be indistinguishable from a connector-derived event once it is normalized: same normalization, same pathway classification, same cross-layer deduplication, same attribution stages, same dashboards, same alerts, same exports.

The consequence matters more than the mechanism. No attribution, scoring, dashboard, alarm, or export capability is allowed to be conditional on having an active programmatic connector. A tenant with zero connectors that imports a supported billing or usage export is specified to reach materialized attribution records and, where the evidence supports it, at least one chargeback-eligible edge.

Formats

The importer is specified to accept two tiers.

Tier Formats
First tier .csv, .tsv, .json, .jsonl / .ndjson, .parquet, .xlsx, each optionally .gz or .zip compressed
Extended tier (later) .avro, .orc, plaintext syslog, CEF, LEEF, and OTLP file exports in JSON or protobuf encoding

An unsupported format is specified to fail with a typed error that names the formats that are supported, rather than a partial or best-effort parse. A compressed file and its uncompressed twin must produce identical deduplication keys, so how you packaged the export cannot change the result.

Source templates

Rather than making every customer describe their file, the importer is specified to ship maintained mapping templates for the exports enterprises actually have:

  • AWS Cost and Usage Report, as a file
  • Azure Cost Management export
  • Google Cloud BigQuery billing export, extracted
  • FOCUS v1.2
  • Anthropic and OpenAI usage and cost exports
  • LiteLLM, Helicone, Portkey, and langfuse gateway logs
  • Bedrock invocation logs
  • The Power Platform consumption report (this closes the previously unspecified ingestion mechanism behind the Power Platform connector)

Each template carries a pin to the upstream schema version it was written against, plus drift tests. When an upstream provider changes its export shape, the contract test is specified to fail before the change reaches production rather than after it has quietly corrupted a month of attribution. A template must ingest its pinned fixture to at least the field coverage the equivalent connector achieves.

Files that match no template

For an unrecognized file, the importer is specified to infer column types and headers and present a mapping preview against the invocation-event field dictionary, with per-field confidence. Mappings are saved, versioned, and scoped to your tenant, so the second file of the same shape does not require re-entry. Required-field completeness is surfaced before you commit, and a mapping that leaves required fields unfilled cannot commit those rows.

Nothing is written before you have seen the report

Every import is specified to run a dry run first, producing a downloadable validation report that names the row, the column, the error code, and a suggested fix, with zero writes. You then explicitly commit the validated subset. Per-row outcomes are reported as created, deduplicated, quarantined, or error, and quarantined rows are exportable so they can be corrected.

The correction loop is specified to be idempotent: you fix the flagged rows or re-upload a corrected file, and the importer reconciles against the prior attempt by stable key, so corrections do not duplicate rows that already committed.

Re-uploads and overlapping periods

Double-counting is the failure mode that destroys trust in a cost number, so the specification addresses it in two places.

  • File level. Each import persists a manifest: SHA-256 digest, byte size, row count, covered time window, and the mapping or template version used. A byte-identical re-upload is specified to be an audited no-op. Re-uploading the same Cost and Usage Report twice must not change any total.
  • Row level. Rows that overlap a window you have already imported collapse on a deterministic source event ID minted during normalization. Two exports that overlap by five days must produce each invocation exactly once, and post-import reconciliation must show zero double-counted cost.

Timestamps and time zones

Every row timestamp is specified to resolve to a time-zone-aware UTC value by one of three routes, in order: a time zone declared in the mapping or template; otherwise an inference that is presented for your explicit confirmation; otherwise rejection. A silent assumption of local time or ingest time is prohibited. Epoch values have their unit detected (seconds, milliseconds, or microseconds) and confirmed.

Period-granular rows, such as daily or monthly billing lines, are specified to carry an explicit period label and to be priced by the event-time pricing rule. A daily-granularity row is never rendered as though it were a point-in-time invocation, and a file whose timestamps are naive cannot commit until a time zone is confirmed. Re-importing under a corrected time zone supersedes the prior rows instead of duplicating them.

What file-derived evidence is worth

File-derived events are specified to carry their own ingestion-layer and source-of-observation markers, with the file manifest ID recorded in the provenance chain, so any materialized figure traces back to the file it came from.

Confidence is not uniform across file sources:

  • Rows ingested through a recognized billing or usage authority template rank as equivalent to cloud-billing-derived evidence. The chargeback-eligible band is reachable for them, under the same operational-confidence floor of 0.80 and the same policy ceiling of 0.95 that apply everywhere else in the product. See Confidence & evidence.
  • Rows from generic or custom mappings are capped as provisional until they are reconciled against an authoritative source. A generic-mapping row is specified never to reach chargeback-ready without that reconciliation.

Totaling must include template-derived rows and must not silently exclude any of them.

Governance dependency

The new ingestion-layer and source-of-observation enum members and the confidence cap ruling above require ratification against the canonical reference before they bind. They are specified, not ratified.

Freshness for a source that arrives as files

A file source declares a cadence: one_shot, ad_hoc, or scheduled(interval). Staleness watermarks, freshness rendering, degradation transitions, and alert or digest bindings are specified to evaluate against the declared cadence rather than against a live-connector assumption.

That distinction is what keeps the product honest about a one-off import. A one_shot source never enters a degraded state and is labeled with the window it covers. A source declared as scheduled monthly alerts only after its interval plus grace has elapsed, not the moment it stops looking fresh.

Large files and import jobs

Uploads are specified to be multipart and resumable up to published per-file size and row ceilings, and imports run asynchronously off the gateway hot path behind an import-job status surface that reports progress and supports cancel and retry. A canceled job must leave zero committed rows. Import throughput is bounded so that concurrent imports cannot breach the platform’s latency and availability objectives, and a bulk import must not regress gateway P99.

The ceilings are not published yet

The requirement is that per-file size and row ceilings are published. It does not fix their values, and no ceiling has been published. Any specific file-size or row-count limit you see attributed to Venturi today is not sourced from the governed requirement. Treat the limits as unstated until they appear here.

What happens to the contents of your file

The no-content-capture posture applies to uploads, and the specification adds parser hardening on top of it.

  • Prompt, completion, and message-body columns are dropped at the ingest edge before persistence. Content capture remains off by default and is only ever possible under an explicit per-workload opt-in, with redaction applied at ingest.
  • A PII scan runs on any retained free-text field.
  • No cell content is ever evaluated as code, a formula, or a path. A formula-injection spreadsheet is specified to be rejected with a typed error and zero partial writes.
  • Parsers enforce decompression-ratio and entity-expansion limits, MIME and type sniffing, and archive-member allowlists. A decompression bomb is rejected, not processed.
  • The upload endpoint is added to the attack-surface inventory with an owning control family and security-gate fixtures, and format fuzzing extends to every supported format.

The worked acceptance case is a gateway log that contains prompt bodies: it imports successfully, and the content columns are absent from every store.

Recurring drops from your own storage

Instead of uploading by hand every month, a tenant admin is specified to be able to register a customer-owned object-storage prefix as a recurring file source. The registration is read-only and is fetched only through the mediated access path that governs every other Venturi integration, so it does not widen the trust boundary.

Discovered files flow through the identical pipeline described on this page, including manifest deduplication, and obey the same cadence contract. Dropping a new monthly Cost and Usage Report into the registered prefix ingests it within the declared interval. Revoking the registration stops ingestion and is audited.

How imported data is labeled in the product

Import-derived data is specified to be visibly distinct from live-captured data, so nobody reads a retrospective number as a live one. Every surface that renders detections, attributions, coverage, or anomalies derived from imported files must display:

  • the acquisition mode,
  • the covered window, stated as-of, and
  • a per-capability availability state: available, degraded with reduced granularity, or requires live capture.

Alerts computed from imports are labeled retrospective. A capability that genuinely needs live capture shows requires live capture, never a score of zero, because a zero would read as a measured finding rather than an absence of data. Import-derived tiles carry no live-freshness affordances, so the live toggle on the invocation feed does not appear on them.

Audit trail

Every import job is specified to be recorded in the audit trail with the actor, the file manifest, the mapping or template version, a reference to the dry-run report, the per-outcome counts, and the commit decision. Import lineage is queryable, the audit-log explorer filters by import job, and a materialized figure traces back to the manifest of the file that produced it.

Onboarding with files alone

File import is specified to appear in the first-run wizard and on the integrations surface as a peer of connecting a source, not as a fallback buried in a settings page. Analytic empty states deep-link into the importer. Where API access has not been granted, representative export files are accepted in place of the connector prerequisites, and the first-attribution milestone has a file-based variant.

Import-only onboarding is a first-class supported path, not an error state, and the end-to-end onboarding objective explicitly counts a tenant that has uploaded one provider export and has no connectors at all. See the 30-minute quickstart for the connector-based path that works today.

What this page deliberately does not tell you

Because the requirements do not state them, this page names no API path, no request or response shape, no CLI command or flag, no configuration key, no screenshot, no provider export filename, and no numeric size, row, or throughput limit. When those are fixed by implementation, they will be documented here and this page’s release-state label will change in the binding index first.