LitPro - Literate Programming Framework

LitPro is a modern literate programming system where the document is primary and the runnable code is derived. It follows the tradition of Knuth's WEB and noweb systems, where code blocks are named and expanded in dependency order.

Unlike notebooks, LitPro produces deterministic output with no hidden state. The narrative order is chosen for human understanding, not execution order.

Literate Programming Philosophy

An ideal Python literate program is not a notebook. It is a single narrative document that derives runnable Python. The document is primary. The script is generated.

Key Principles

  1. The narrative order is chosen for human understanding
  2. Code chunks are named and reusable
  3. Tangling generates a .py file in dependency order
  4. No hidden state exists outside what is shown
  5. Running the generated script yields identical behavior every time

Comparison with Notebooks

Aspect Literate Programming (LPMD) Notebook (Jupyter)
Primary artifact Document with narrative Executable cells
Execution model Deterministic, dependency-ordered Interactive, cell-by-cell
State management No hidden state Mutable kernel state
Output reproducibility Identical every time Depends on execution history
Document structure Narrative-driven Linear execution flow

Example: Spectral Envelope Estimation

Here's how a literate program might look:

---
Title: Spectral Envelope Estimation in Real-Time
---

We begin by defining the signal model. The input signal is assumed to be stationary over a short window and represented as a discrete sequence x[n].

We require a window function:

```python
def hann(N: int) -> np.ndarray:
return 0.5 - 0.5 * np.cos(2 * np.pi * np.arange(N) / N)
```

Next we define the STFT. The important property is overlap-add consistency.

```python
def stft(x: np.ndarray, N: int, hop: int) -> np.ndarray:
w = hann(N)
frames = []
for i in range(0, len(x) - N, hop):
frames.append(np.fft.rfft(x[i:i+N] * w))
return np.stack(frames)
```

We now define the envelope estimator…

```python
def spectral_envelope(X: np.ndarray) -> np.ndarray:
return np.abs(X).mean(axis=0)
```

Finally, we provide a CLI entry point:

```python
if __name__ == "__main__":
import soundfile as sf
x, sr = sf.read("input.wav")
X = stft(x, 2048, 512)
env = spectral_envelope(X)
print(env)
```

LPMD Syntax

LPMD uses HTML comments as cell markers to keep the markdown readable while enabling execution. These markers allow for dependency management similar to noweb's named chunks:

Basic Cell

<!-- cell:cell_id -->
```python
print("Hello from LPMD!")
x = 10
```

Cell with Dependencies

<!-- cell:compute depends:setup -->
```python
result = x * 2
print(f"Result: {result}")
```

Reusable Code Block

<!-- cell:window-function -->
```python
def hann(N):
return 0.5 - 0.5 * np.cos(2 * np.pi * np.arange(N) / N)
```

Parameters:

During execution, LPMD resolves dependencies using topological sorting, ensuring that code blocks are evaluated in the correct order regardless of their position in the narrative.

LitPro Runner

Try executing LitPro code directly in your browser. Enter your LitPro code below and click Run:

How to Use LitPro

Installation

pip install litpro

Execute File

litpro run myfile.lit

Generate HTML

litpro html myfile.lit

Workflow

  1. Write your narrative document with named code blocks
  2. Specify dependencies between blocks using the depends parameter
  3. LitPro resolves execution order using topological sorting
  4. Execute the document to verify correctness
  5. Generate clean code for deployment

This workflow ensures that your code is both well-documented and functionally correct, following the true spirit of literate programming.

Why LPMD Instead of Jupyter/Colab?

While Jupyter and Colab are popular tools, they have fundamental limitations that make them unsuitable for true literate programming:

Major Issues with Jupyter/Colab

Hidden State

Jupyter notebooks maintain mutable kernel state. Cell execution order affects results, making notebooks non-reproducible. You can run cells in any order, leading to inconsistent states that are hard to reproduce.

Non-Deterministic Execution

Results depend on execution history. A cell that worked yesterday might fail today if previous cells were modified or skipped. This makes debugging and verification difficult.

Linear Narrative

Notebooks force a top-down narrative that may not match the logical flow of ideas. The presentation order is tied to execution order, limiting pedagogical flexibility.

Version Control Problems

Notebooks contain binary outputs mixed with code, causing massive diffs in version control. Collaboration becomes difficult as output changes clutter commits.

Hidden Dependencies

Cells often rely on variables from other cells without explicit declaration. This creates fragile dependencies that break silently when cells are reordered or deleted.

Not Real Programs

Notebooks are not directly executable as programs. Converting to scripts requires manual extraction and reorganization, often breaking the logical flow.

LPMD Advantages

Simple Commands

Installation

pip install litpro

Execute File

litpro run myfile.lit

Generate Script

litpro export myfile.lit

Generate HTML

litpro html myfile.lit

These simple commands make it easy to integrate LitPro into your development workflow.

Embedding in Websites

Embed LitPro execution directly in your website or blog:

<script src="https://cdn.jsdelivr.net/npm/litpro-web-component@latest/litpro-runner.js"></script>
<litpro-runner>
  <pre><code>
    <!-- cell:setup -->
    ```python
    print("Hello from LitPro!")
    x = 10
    ```
  </code></pre>
</litpro-runner>

Or use the JavaScript API:

<script>
  import { LitPro } from 'litpro-web-component';

  const litpro = new LitPro({
    selector: '#litpro-container',
    code: `<!-- cell:setup -->
      `python
      print("Hello from LitPro!")
      x = 10
      ```
      <!-- cell:compute depends:setup -->
      `python
      result = x * 2
      print(f"Result: {result}")
      ```
  });

  litpro.render();
</script>

This allows readers to execute your literate programs directly in their browsers.

Adapting to Other Languages

LitPro's architecture makes it easy to extend to other programming languages:

Language Adapters

Each language requires a simple adapter that implements:

  • Code parser for the language
  • Dependency resolver
  • Execution environment
  • Error formatter

Example: JavaScript Adapter

// litpro-js-adapter.js
export class JavaScriptAdapter {
  parse(code) { /* parse JS code */ }
  execute(code, context) { /* execute JS */ }
  resolveDependencies(cells) { /* resolve deps */ }
}

Supported Languages

Planned adapters for:

  • JavaScript/TypeScript
  • Rust
  • Go
  • Julia
  • R
  • C/C++

The same literate programming principles apply across all languages, making it easy to switch between ecosystems while maintaining the same workflow.