Runix Pipeline is managed data preparation: we turn raw, duplicated, half-structured source material into something a model can read. It is delivered as an engagement, not self-serve software: this page explains what an engagement looks like so you know what to expect before you write to us. Pipeline is being built with design partners, so early projects get direct engineering attention.
1. Scoping
Every engagement starts with your data and your goal. You describe the sources (documents, exports, crawls, databases), roughly how much there is, and what the output feeds: analytics, retrieval, or model training. We come back with a written scope: stages to run, delivery format, timeline and price. Scope is agreed before work starts and changes are agreed in writing.
2. The six stages
- Ingest: pulling from your sources into a working store, with provenance kept per record.
- Clean & dedupe: format normalisation and noise reduction, then deduplication at exact, near-duplicate and semantic tiers, applied to the depth the use case needs.
- Structure: schema-validated extraction from unstructured text, so downstream code gets fields, not prose.
- Mask: personal information identified and masked before data travels; the masking step fails closed.
- Report: coverage, duplication rates, extraction confidence, and what was dropped and why. Part of the deliverable, not an extra.
- Deliver: dataset files, database tables or an endpoint, batch or continuous, whichever your downstream actually consumes.
Not every project needs every stage: the scope names which ones run and why. The background reasoning is in Building AI data pipelines.
3. Delivery
The last two stages above are what you actually receive: the processed dataset in the agreed format — dataset files, database tables or an API handoff — and the quality report that goes with it, covering what was ingested, what was dropped and why, duplication rates, extraction validation results and masking coverage.
4. Starting a project
Tell us about your data — sources, volume, and what it needs to feed. We reply within one business day; if the fit is right, the next step is a written scope.