RTHDDNetComposer

A high-performance neural network compositor with dual-language architecture (C# for graph construction, C++ for runtime execution) using memory-mapped files for zero-copy communication.

Status: Phase 3 MVP + Phase 3D Partial + Phase 4A Complete ✅ (622 C# + 114 C++ unit + 17 C++ integration tests passing)

Overview

RTHDDNetComposer implements a unique dual-graph architecture where forward computation and gradient flow exist as separate, parallel computational graphs. This design enables:

Multiple gradient sources: Rewards, supervised loss, intrinsic curiosity
Complex gradient routing: Independent modification of gradient flow without changing forward computation
Zero-copy execution: Memory-mapped files eliminate serialization overhead
Real-time performance: < 16.67ms per timestep for 60 FPS operation

Three-Stage Model Lifecycle

Model Design (C#): Pure topology descriptions - graph structure, node specifications, hyperparameters
Model Package (C#): Complete model = topology + trained weights
Unfolded Memory Map (C++): Runtime-ready form with all buffers pre-allocated

Architecture

C# Layer (Graph Builder)

Declarative graph construction using fluent API
Memory analysis and layout optimization
"Unfolding" process: transforms graphs into memory-mapped runtime representation
LINQ-based port operations for flexible connection patterns

C++ Layer (Execution Runtime)

Opens memory-mapped files created by C#
Zero-copy direct buffer access
Lock-free circular buffers for I/O streaming
Thread pool with CPU affinity control
Gradient signal routing and processing

Numeric Domains

Support for multiple numeric domains:

Real: Standard floating-point
Complex: Frequency domain processing
Quaternion: 3D rotations and spatial relationships
Octonion: Higher-dimensional algebraic structures

Current Phase Status

Completed Phases

✅ Phase 0 (Foundation): Core type system (C# + C++)
✅ Phase 1 (Graph Construction): Complete builder API with InstantNode/TemporalNode architecture
✅ Phase 1A (Constrained Types): Extended type system with unit-constrained domains
✅ Phase 1.5 (Visualization + Node Identity): Graph visualization, diagnostic output, GUID-based node identity
✅ Phase 2A-D (Memory Mapping): .mmap file generation with topology serialization
✅ Phase 3A-C (C++ Execution Engine): Memory-mapped graph loading, buffer access, execution engine
✅ Phase 3D (Partial): IntegratorNode temporal execution (LTCNode deferred)
✅ Phase 4A (Gradient Graph Construction): Complete C# gradient graph API

Phase 3D Achievements

IntegratorNode - First temporal/stateful node in C++ runtime:

Exponential moving average: s(t+1) = (1-α)·s(t) + α·gain·input
State buffer allocation and initialization
Fixed critical MemoryAnalyzer bug (state buffer naming)
11 unit tests + 4 integration tests
Mathematical correctness verified (convergence, tracking, state persistence)

Deferred to Future: LTCNode, conversion nodes (C++ execution), multi-threading, performance validation

Phase 4A Achievements

Gradient Graph Construction API - Dual-graph architecture foundation:

4 gradient source types (RewardSignal, SupervisedLoss, IntrinsicCuriosity, RegularizationLoss)
5 gradient transforms (Scale, Clip, Normalize, Gate, Accumulate)
4 gradient routing modes (Direct, Broadcast, Accumulate, Conditional)
GradientGraphBuilder fluent API for declarative gradient graph construction
Complete validation (DAG enforcement, cycle detection, orphan detection)
Forward-gradient mapping via node GUIDs
106 new tests (all passing)

Next Phase

Phase 4B (Gradient Memory Mapping): Extend .mmap format to serialize gradient graphs for C++ runtime

Prerequisites

Required

.NET SDK 8.0 or later
CMake 3.18 or later
C++20 compatible compiler (GCC 11+, Clang 13+, or MSVC 2022+)
Git

Optional

CUDA Toolkit 12.x (for GPU acceleration)
Ninja (faster C++ builds)
VS Code (recommended IDE)

Quick Start

1. Clone the Repository

git clone https://github.com/rjogilvie/RTHDDNetComposer.git
cd RTHDDNetComposer

2. Check Environment

# Run environment setup script
./scripts/setup_env.sh

# Check for CUDA (optional)
./scripts/detect_cuda.sh

3. Build C# Projects

# Restore dependencies
dotnet restore RTHDDNetComposer.sln

# Build all C# projects
dotnet build RTHDDNetComposer.sln --configuration Release

# Run tests
dotnet test RTHDDNetComposer.sln

4. Build C++ Projects

# Configure (without CUDA)
mkdir -p build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release -DENABLE_CUDA=OFF

# Build
cmake --build . -j$(nproc)

# Run tests
ctest --output-on-failure

cd ..

5. Build with CUDA (Optional)

# Configure with CUDA
mkdir -p build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release -DENABLE_CUDA=ON

# Build
cmake --build . -j$(nproc)

cd ..

Project Structure

RTHDDNetComposer/
├── src/
│   ├── NetComposer.Core/         # C# core data structures
│   ├── NetComposer.Builder/      # C# graph building API
│   ├── NetComposer.MemoryMap/    # C# memory mapping
│   ├── NetComposer.Simulation/   # C# simulation interfaces
│   └── cpp/                      # C++ runtime
│       ├── core/                 # Memory-mapped graph, buffers
│       ├── execution/            # Execution engine, threading
│       ├── gradient/             # Gradient processing
│       ├── simulation/           # Simulation adapters
│       └── io/                   # I/O controller
├── include/                      # C++ public headers
├── tests/
│   ├── NetComposer.Tests/        # C# NUnit tests
│   └── cpp/                      # C++ Google Test tests
├── docs/                         # Architecture documentation
├── scripts/                      # Build and setup scripts
├── .github/workflows/            # CI/CD pipelines
└── CLAUDE.md                     # AI assistant guidance

Development Workflow

Using VS Code

Open the project in VS Code
Install recommended extensions (prompted automatically)
Build tasks available via Ctrl+Shift+B / Cmd+Shift+B
Debug configurations in Run and Debug panel

Build Tasks (VS Code)

Build C# (Debug/Release): Compile C# solution
Build C++ (Debug/Release): Configure and compile C++ projects
Test C#: Run NUnit tests
Test C++: Run Google Test suite
Build All: Build both C# and C++ projects

Command Line

# C# development
dotnet build                      # Build
dotnet test                       # Run tests
dotnet run --project <project>    # Run specific project

# C++ development
cmake -B build -S .               # Configure
cmake --build build               # Build
cd build && ctest                 # Test

# Clean builds
rm -rf build                      # Clean C++
dotnet clean                      # Clean C#

Testing

C# Tests (NUnit)

# Run all tests
dotnet test

# Run with detailed output
dotnet test --logger "console;verbosity=detailed"

# Run specific test
dotnet test --filter "TestName~<pattern>"

C++ Tests (Google Test)

cd build

# Run all tests
ctest --output-on-failure

# Run specific test
./tests/cpp/netcomposer_tests --gtest_filter="CoreTests.*"

Benchmarking

RTHDDNetComposer includes a comprehensive benchmark suite for performance validation and regression detection.

Running All Benchmarks

# Run complete benchmark suite (C++ + C#)
./scripts/run_benchmarks.sh

Results Location:

C++ benchmarks: benchmarks/results/cpp_benchmarks.json
C# benchmarks: benchmarks/results/csharp/
Baseline report: benchmarks/baseline_report.md

Performance Baseline

C++ Runtime (24 benchmarks):

Simple timestep: 84 ns (11.9M timesteps/sec)
10-layer network: 1.34 μs (749K timesteps/sec)
Buffer registry lookup: 13.9 ns (72.1M lookups/sec)
Gradient computation (BPTT): 329 ns (3.04M timesteps/sec)
Scalability: O(n) complexity validated up to 10,000 nodes

Key Findings:

✅ All benchmarks meet 60 FPS target (< 16.67 ms per timestep)
✅ Up to 4,000-node graphs supported at 60 FPS
✅ 10,000-node graphs run at 25 FPS (38.9 ms per timestep)

For detailed results, see benchmarks/baseline_report.md.

Running Individual Benchmark Suites

C++ Only:

cmake -B build -DCMAKE_BUILD_TYPE=Release -DBUILD_BENCHMARKS=ON
cmake --build build --target netcomposer_benchmarks -j$(nproc)
cd build/benchmarks/cpp && ./netcomposer_benchmarks

C# Only:

cd benchmarks/NetComposer.Benchmarks
dotnet run -c Release

See benchmarks/README.md for complete benchmarking guide.

Documentation

CLAUDE.md - Guidance for AI assistants working with this codebase
docs/ml-compositor-api-spec.md - Complete C# API specification
docs/memory-mapping-architecture.md - Memory layout and lifecycle
docs/memory-mapped-implementation.md - C++ runtime implementation
docs/compositor-data-structures.md - Core data structures
docs/simulation-io-architecture.md - Simulation adapter interfaces
docs/testing-plan-document.md - Testing strategy

Performance Targets

Average timestep: < 16.67ms (60 FPS)
Worst-case latency: < 20ms
Jitter: < 2ms standard deviation
Cache hit rate: > 90%
Zero runtime allocations in execution loop

Contributing

Code Style

C#: Follow .NET conventions (enforced by EditorConfig)
C++: C++20 standard, follow project style (enforced by .editorconfig)
Commits: Conventional commits format

Pull Requests

Fork the repository
Create a feature branch (git checkout -b feature/amazing-feature)
Make your changes with clear commit messages
Run tests locally
Push to your fork
Open a Pull Request

Continuous Integration

GitHub Actions workflows automatically:

Build C# and C++ projects
Run test suites
Check documentation
Validate code style

See .github/workflows/ for CI configuration.

License

[Specify your license here]

Authors

Robert Ogilvie - @rjogilvie

Acknowledgments

See docs/ for detailed architecture documentation and design decisions.

Troubleshooting

CUDA Not Found

Run the CUDA detection script:

./scripts/detect_cuda.sh

Follow the instructions to install CUDA or build without CUDA support using -DENABLE_CUDA=OFF.

CMake Configuration Issues

# Clean CMake cache
rm -rf build
mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Debug

.NET SDK Version Mismatch

Check global.json for pinned SDK version:

cat global.json
dotnet --list-sdks

Install the matching SDK version or update global.json.

Support

Issues: GitHub Issues
Documentation: docs/
CLAUDE.md: Guidance for AI assistants

RTHDDNetComposer

A high-performance neural network compositor with dual-language architecture (C# for graph construction, C++ for runtime execution) using memory-mapped files for zero-copy communication.

Status: Phase 3 MVP + Phase 3D Partial + Phase 4A Complete ✅ (622 C# + 114 C++ unit + 17 C++ integration tests passing)

Overview

RTHDDNetComposer implements a unique dual-graph architecture where forward computation and gradient flow exist as separate, parallel computational graphs. This design enables:

Multiple gradient sources: Rewards, supervised loss, intrinsic curiosity
Complex gradient routing: Independent modification of gradient flow without changing forward computation
Zero-copy execution: Memory-mapped files eliminate serialization overhead
Real-time performance: < 16.67ms per timestep for 60 FPS operation

Three-Stage Model Lifecycle

Model Design (C#): Pure topology descriptions - graph structure, node specifications, hyperparameters
Model Package (C#): Complete model = topology + trained weights
Unfolded Memory Map (C++): Runtime-ready form with all buffers pre-allocated

Architecture

C# Layer (Graph Builder)

Declarative graph construction using fluent API
Memory analysis and layout optimization
"Unfolding" process: transforms graphs into memory-mapped runtime representation
LINQ-based port operations for flexible connection patterns

C++ Layer (Execution Runtime)

Opens memory-mapped files created by C#
Zero-copy direct buffer access
Lock-free circular buffers for I/O streaming
Thread pool with CPU affinity control
Gradient signal routing and processing

Numeric Domains

Support for multiple numeric domains:

Real: Standard floating-point
Complex: Frequency domain processing
Quaternion: 3D rotations and spatial relationships
Octonion: Higher-dimensional algebraic structures

Current Phase Status

Completed Phases

✅ Phase 0 (Foundation): Core type system (C# + C++)
✅ Phase 1 (Graph Construction): Complete builder API with InstantNode/TemporalNode architecture
✅ Phase 1A (Constrained Types): Extended type system with unit-constrained domains
✅ Phase 1.5 (Visualization + Node Identity): Graph visualization, diagnostic output, GUID-based node identity
✅ Phase 2A-D (Memory Mapping): .mmap file generation with topology serialization
✅ Phase 3A-C (C++ Execution Engine): Memory-mapped graph loading, buffer access, execution engine
✅ Phase 3D (Partial): IntegratorNode temporal execution (LTCNode deferred)
✅ Phase 4A (Gradient Graph Construction): Complete C# gradient graph API

Phase 3D Achievements

IntegratorNode - First temporal/stateful node in C++ runtime:

Exponential moving average: s(t+1) = (1-α)·s(t) + α·gain·input
State buffer allocation and initialization
Fixed critical MemoryAnalyzer bug (state buffer naming)
11 unit tests + 4 integration tests
Mathematical correctness verified (convergence, tracking, state persistence)

Deferred to Future: LTCNode, conversion nodes (C++ execution), multi-threading, performance validation

Phase 4A Achievements

Gradient Graph Construction API - Dual-graph architecture foundation:

4 gradient source types (RewardSignal, SupervisedLoss, IntrinsicCuriosity, RegularizationLoss)
5 gradient transforms (Scale, Clip, Normalize, Gate, Accumulate)
4 gradient routing modes (Direct, Broadcast, Accumulate, Conditional)
GradientGraphBuilder fluent API for declarative gradient graph construction
Complete validation (DAG enforcement, cycle detection, orphan detection)
Forward-gradient mapping via node GUIDs
106 new tests (all passing)

Next Phase

Phase 4B (Gradient Memory Mapping): Extend .mmap format to serialize gradient graphs for C++ runtime

Prerequisites

Required

.NET SDK 8.0 or later
CMake 3.18 or later
C++20 compatible compiler (GCC 11+, Clang 13+, or MSVC 2022+)
Git

Optional

CUDA Toolkit 12.x (for GPU acceleration)
Ninja (faster C++ builds)
VS Code (recommended IDE)

Quick Start

1. Clone the Repository

git clone https://github.com/rjogilvie/RTHDDNetComposer.git
cd RTHDDNetComposer

2. Check Environment

# Run environment setup script
./scripts/setup_env.sh

# Check for CUDA (optional)
./scripts/detect_cuda.sh

3. Build C# Projects

# Restore dependencies
dotnet restore RTHDDNetComposer.sln

# Build all C# projects
dotnet build RTHDDNetComposer.sln --configuration Release

# Run tests
dotnet test RTHDDNetComposer.sln

4. Build C++ Projects

# Configure (without CUDA)
mkdir -p build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release -DENABLE_CUDA=OFF

# Build
cmake --build . -j$(nproc)

# Run tests
ctest --output-on-failure

cd ..

5. Build with CUDA (Optional)

# Configure with CUDA
mkdir -p build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release -DENABLE_CUDA=ON

# Build
cmake --build . -j$(nproc)

cd ..

Project Structure

RTHDDNetComposer/
├── src/
│   ├── NetComposer.Core/         # C# core data structures
│   ├── NetComposer.Builder/      # C# graph building API
│   ├── NetComposer.MemoryMap/    # C# memory mapping
│   ├── NetComposer.Simulation/   # C# simulation interfaces
│   └── cpp/                      # C++ runtime
│       ├── core/                 # Memory-mapped graph, buffers
│       ├── execution/            # Execution engine, threading
│       ├── gradient/             # Gradient processing
│       ├── simulation/           # Simulation adapters
│       └── io/                   # I/O controller
├── include/                      # C++ public headers
├── tests/
│   ├── NetComposer.Tests/        # C# NUnit tests
│   └── cpp/                      # C++ Google Test tests
├── docs/                         # Architecture documentation
├── scripts/                      # Build and setup scripts
├── .github/workflows/            # CI/CD pipelines
└── CLAUDE.md                     # AI assistant guidance

Development Workflow

Using VS Code

Open the project in VS Code
Install recommended extensions (prompted automatically)
Build tasks available via Ctrl+Shift+B / Cmd+Shift+B
Debug configurations in Run and Debug panel

Build Tasks (VS Code)

Build C# (Debug/Release): Compile C# solution
Build C++ (Debug/Release): Configure and compile C++ projects
Test C#: Run NUnit tests
Test C++: Run Google Test suite
Build All: Build both C# and C++ projects

Command Line

# C# development
dotnet build                      # Build
dotnet test                       # Run tests
dotnet run --project <project>    # Run specific project

# C++ development
cmake -B build -S .               # Configure
cmake --build build               # Build
cd build && ctest                 # Test

# Clean builds
rm -rf build                      # Clean C++
dotnet clean                      # Clean C#

Testing

C# Tests (NUnit)

# Run all tests
dotnet test

# Run with detailed output
dotnet test --logger "console;verbosity=detailed"

# Run specific test
dotnet test --filter "TestName~<pattern>"

C++ Tests (Google Test)

cd build

# Run all tests
ctest --output-on-failure

# Run specific test
./tests/cpp/netcomposer_tests --gtest_filter="CoreTests.*"

Benchmarking

RTHDDNetComposer includes a comprehensive benchmark suite for performance validation and regression detection.

Running All Benchmarks

# Run complete benchmark suite (C++ + C#)
./scripts/run_benchmarks.sh

Results Location:

C++ benchmarks: benchmarks/results/cpp_benchmarks.json
C# benchmarks: benchmarks/results/csharp/
Baseline report: benchmarks/baseline_report.md

Performance Baseline

C++ Runtime (24 benchmarks):

Simple timestep: 84 ns (11.9M timesteps/sec)
10-layer network: 1.34 μs (749K timesteps/sec)
Buffer registry lookup: 13.9 ns (72.1M lookups/sec)
Gradient computation (BPTT): 329 ns (3.04M timesteps/sec)
Scalability: O(n) complexity validated up to 10,000 nodes

Key Findings:

✅ All benchmarks meet 60 FPS target (< 16.67 ms per timestep)
✅ Up to 4,000-node graphs supported at 60 FPS
✅ 10,000-node graphs run at 25 FPS (38.9 ms per timestep)

For detailed results, see benchmarks/baseline_report.md.

Running Individual Benchmark Suites

C++ Only:

cmake -B build -DCMAKE_BUILD_TYPE=Release -DBUILD_BENCHMARKS=ON
cmake --build build --target netcomposer_benchmarks -j$(nproc)
cd build/benchmarks/cpp && ./netcomposer_benchmarks

C# Only:

cd benchmarks/NetComposer.Benchmarks
dotnet run -c Release

See benchmarks/README.md for complete benchmarking guide.

Documentation

CLAUDE.md - Guidance for AI assistants working with this codebase
docs/ml-compositor-api-spec.md - Complete C# API specification
docs/memory-mapping-architecture.md - Memory layout and lifecycle
docs/memory-mapped-implementation.md - C++ runtime implementation
docs/compositor-data-structures.md - Core data structures
docs/simulation-io-architecture.md - Simulation adapter interfaces
docs/testing-plan-document.md - Testing strategy

Performance Targets

Average timestep: < 16.67ms (60 FPS)
Worst-case latency: < 20ms
Jitter: < 2ms standard deviation
Cache hit rate: > 90%
Zero runtime allocations in execution loop

Contributing

Code Style

C#: Follow .NET conventions (enforced by EditorConfig)
C++: C++20 standard, follow project style (enforced by .editorconfig)
Commits: Conventional commits format

Pull Requests

Fork the repository
Create a feature branch (git checkout -b feature/amazing-feature)
Make your changes with clear commit messages
Run tests locally
Push to your fork
Open a Pull Request

Continuous Integration

GitHub Actions workflows automatically:

Build C# and C++ projects
Run test suites
Check documentation
Validate code style

See .github/workflows/ for CI configuration.

License

[Specify your license here]

Authors

Robert Ogilvie - @rjogilvie

Acknowledgments

See docs/ for detailed architecture documentation and design decisions.

Troubleshooting

CUDA Not Found

Run the CUDA detection script:

./scripts/detect_cuda.sh

Follow the instructions to install CUDA or build without CUDA support using -DENABLE_CUDA=OFF.

CMake Configuration Issues

# Clean CMake cache
rm -rf build
mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Debug

.NET SDK Version Mismatch

Check global.json for pinned SDK version:

cat global.json
dotnet --list-sdks

Install the matching SDK version or update global.json.

Support

Issues: GitHub Issues
Documentation: docs/
CLAUDE.md: Guidance for AI assistants

rjogilvie/NetComposer.MemoryMapv0.1.0

Get Started

Readme

RTHDDNetComposer

Overview

Three-Stage Model Lifecycle

Architecture

C# Layer (Graph Builder)

C++ Layer (Execution Runtime)

Numeric Domains

Current Phase Status

Completed Phases

Phase 3D Achievements

Phase 4A Achievements

Next Phase

Prerequisites

Required

Optional

Quick Start

1. Clone the Repository

2. Check Environment

3. Build C# Projects

4. Build C++ Projects

5. Build with CUDA (Optional)

Project Structure

Development Workflow

Using VS Code

Build Tasks (VS Code)

Command Line

Testing

C# Tests (NUnit)

C++ Tests (Google Test)

Benchmarking

Running All Benchmarks

Performance Baseline

Running Individual Benchmark Suites

Documentation

Performance Targets

Contributing

Code Style

Pull Requests

Continuous Integration

License

Authors

Acknowledgments

Troubleshooting

CUDA Not Found

CMake Configuration Issues

.NET SDK Version Mismatch

Support

rjogilvie/NetComposer.MemoryMapv0.1.0

Get Started

Readme

RTHDDNetComposer

Overview

Three-Stage Model Lifecycle

Architecture

C# Layer (Graph Builder)

C++ Layer (Execution Runtime)

Numeric Domains

Current Phase Status

Completed Phases

Phase 3D Achievements

Phase 4A Achievements

Next Phase

Prerequisites

Required

Optional

Quick Start

1. Clone the Repository

2. Check Environment

3. Build C# Projects

4. Build C++ Projects

5. Build with CUDA (Optional)

Project Structure

Development Workflow

Using VS Code

Build Tasks (VS Code)

Command Line

Testing