compiled classical machine learning / Flow

Rebuild the estimator stack. Measure what actually changes.

flow-scikit reimplements classical ML in Flow and tests it against scikit-learn with parity-gated timings, explicit execution-substrate analysis, native deployment and reproducible benchmark artifacts.

  • 19/19 parity eligible
  • native binary
  • substrate-aware appraisal
canonical parity19 / 19

Every canonical row has resolved parity and measurement status.

Flow timing wins8 / 19

Current canonical end-to-end result; all rows remain visible.

estimator operations mapped491

Generated sklearn execution-substrate inventory.

runtime profiles32

Mixed-stack attribution rows feeding the optimization roadmap.

the important distinction

scikit-learn is not “Python code” in one performance sense.

Its public interface is Python, but hot paths may execute in NumPy/SciPy, BLAS/LAPACK, sklearn-owned compiled code, or external native libraries such as liblinear and libsvm. The meaningful question is therefore not simply whether Flow beats Python, but which execution layer Flow is replacing, retaining, or compiling around.

Inspect the execution map →
Python-boundInterpreter and orchestration work can be direct compilation targets.
Mixed / boundary-heavyWhole-estimator compilation can remove crossings and temporary allocations.
BLAS / LAPACK-boundRetain mature kernels unless evidence supports replacement.
sklearn-owned nativeBenchmark Flow against the compiled implementation directly.
External nativeliblinear/libsvm are native competitors, not interpreted baselines.

current appraisal

The architecture result is more useful than a blanket speed claim.

The current committed map groups all 19 canonical rows by sklearn fit substrate. Flow wins 75% of the rows classified Python-bound, about 45% of mixed rows, and 0% of external-native-bound rows. That is evidence that execution substrate is predictive enough to guide optimization work, while still being far from a causal proof by itself.

What the evidence supports today: Flow has strong end-to-end wins on several classical-ML workloads and a much smaller native deployment model, while mature native solver paths remain serious baselines. The roadmap now ranks work from measured substrate, runtime attribution, benchmark results and implementation readiness rather than from source-language folklore.

evidence chain

Correctness → timing → substrate → attribution → roadmap.

The repository now keeps each stage machine-readable and reproducible.

01 / parity

Resolve all 19 canonical rows.

Competitive timing only follows estimator-specific numerical checks.

Parity evidence →
02 / architecture

Map what sklearn actually executes.

491 estimator-operation rows are classified with evidence and drift detection.

Architecture map →
03 / priorities

Turn evidence into an optimization queue.

Runtime profiles, native-hotspot dispositions and whole-estimator experiments feed a generated roadmap.

Roadmap ↗

minimal example

Fit, predict, inspect.

The library remains a native Flow implementation rather than a Python compatibility layer.

import "lib/scikit/scikit.flow"

let model = knn_classifier_fit(X_train, y_train, 5)
let predictions = knn_classifier_predict(model, X_test)
let score = accuracy_score(y_test, predictions, n_test)

open evidence

Read the benchmark. Inspect the substrate. Reproduce the result.

Open flow-scikit on GitHub ↗