Digital Resource Integrity - High-Level Requirements

Problem Statement

The nondominium system must provide cryptographic proof that downloaded resource data is identical to the original data uploaded to the Holochain DHT. This ensures trust in resource exchange and prevents unauthorized modifications during storage and transmission.

Core Requirements

R1: Cryptographic Integrity Verification

The system shall provide mathematical proof that downloaded data matches the original uploaded data.

Acceptance Criteria:

  • Users can verify that any downloaded resource is identical to the original
  • Verification process works even if data is stored by multiple unknown agents
  • Any modification to data is immediately detectable
  • Verification can be performed without trusting storage providers

R2: Distributed Storage with Guarantees

The system shall store resource data across multiple Holochain agents while maintaining integrity.

Acceptance Criteria:

  • Files are split into chunks for distributed storage
  • Each chunk is cryptographically addressed by its content hash
  • Data can be retrieved even if some storage agents are offline
  • Storage failures do not compromise data integrity

R3: Efficient Verification Process

The system shall allow users to verify resource integrity efficiently.

Acceptance Criteria:

  • Verification can be performed without downloading entire resources
  • Users can verify specific portions of large resources
  • Verification process completes within acceptable time limits
  • Users receive clear success/failure feedback on verification

R4: Manifest-Based Organization

The system shall use manifests to organize and describe resource data.

Acceptance Criteria:

  • Each resource has a manifest describing its structure and integrity
  • Manifests contain metadata about files and their organization
  • Manifests are themselves cryptographically verifiable
  • Manifest format supports multiple files and complex resources

R5: Composable Resource Architecture

The system shall support hierarchical resource compositions where parent resources are assembled from child resources.

Acceptance Criteria:

  • Resources can reference other resources as components with specified quantities
  • Each component maintains independent integrity verification and trust status
  • Parent resource integrity depends on component integrity with dependency rules
  • Component replacements trigger appropriate re-verification of affected parent assemblies
  • Support for required, optional, and alternative components with different verification levels

R6: Fractal Verification Process

The system shall provide recursive verification that cascades through resource hierarchies.

Acceptance Criteria:

  • Verify complete resource tree from leaf components to root assembly
  • Identify specifically which components failed verification in hierarchical context
  • Allow partial verification of subsystems without full assembly verification
  • Component integrity changes propagate trust/verification status up through parent assemblies
  • Support for "trusted component" status that simplifies verification of dependent assemblies

R7: Component Dependency Management

The system shall manage dependencies between components with version compatibility and substitution rules.

Acceptance Criteria:

  • Version compatibility matrices defining which component versions can work together
  • Required vs optional vs alternative component classification with clear rules
  • Component substitution rules that specify verification implications for replacements
  • Assembly upgrade paths that maintain verification continuity and trust propagation

R8: Selective Component Verification

The system shall allow users to verify specific components or subsystems efficiently.

Acceptance Criteria:

  • Verify only critical components for time-sensitive operations with reduced scope
  • Component-level verification status caching to avoid redundant verifications
  • Trust propagation from previously verified components to parent assemblies
  • Impact analysis showing which parent assemblies are affected by component changes

R9: Component Supply Chain Transparency

The system shall provide transparency into component origins, manufacturing processes, and assembly relationships.

Acceptance Criteria:

  • Component provenance tracking through complete assembly hierarchy
  • Manufacturer and supplier verification at component level with cryptographic proofs
  • Integration with external supply chain verification systems and standards
  • Component recall and replacement notifications that affect dependent assemblies

Verification Process Requirements

Phase 1: Resource Discovery & Composition Analysis

User Action: Request a resource from the DHT

System Responsibilities:

  1. Retrieve the resource manifest from the DHT
  2. Verify the manifest's cryptographic signature
  3. Analyze resource composition:
    • Identify if resource is atomic or composite
    • Load component manifests recursively for composite resources
    • Build complete component dependency tree
    • Check version compatibility between components
  4. Present comprehensive manifest metadata including:
    • File list, sizes, and hashes
    • Component breakdown with integrity status
    • Dependency relationships and version constraints
    • Critical vs optional component classification
  5. Obtain user consent to proceed with download

Phase 1A: Component Integrity Pre-Check (Composite Resources)

System Responsibilities:

  1. Verify availability of all required components
  2. Check component verification status and trust levels
  3. Identify any missing, corrupted, or incompatible components
  4. Present component integrity summary before download
  5. Allow user to proceed with available components or wait for missing ones

Phase 2: Hierarchical Chunk Verification During Download

User Action: Download the resource data

System Responsibilities:

  1. For atomic resources: Retrieve and verify individual chunks as before
  2. For composite resources:
    • Download component resources in dependency order
    • For each component: verify its chunks and integrity according to its verification level
    • Track component verification status independently
  3. Real-time integrity feedback:
    • Show component-by-component verification progress
    • Flag individual component failures without stopping entire download
    • Continue with available components while troubleshooting failed ones
  4. Dependency validation:
    • Verify component compatibility during assembly
    • Check that required components are present before marking assembly complete
    • Handle optional component absences gracefully

Phase 3: Complete Assembly Verification

User Action: After download completes, request full verification

System Responsibilities:

  1. For atomic resources: Verify complete resource against manifest's Merkle root
  2. For composite resources:
    • Verify individual component integrity (already done in Phase 2)
    • Verify assembly integrity:
      • Validate component quantities and relationships
      • Check assembly rules and constraints
      • Verify no required components are missing
      • Validate optional component selections
  3. Generate hierarchical verification results:
    • ✅ FULLY VERIFIED: All components and assembly validated
    • ⚠️ PARTIALLY VERIFIED: Core components verified, optional issues present
    • ❌ ASSEMBLY FAILED: Critical components missing or incompatible
    • ❌ COMPONENT CORRUPTED: One or more components failed verification
  4. Detailed component reporting:
    • Show exact component verification status
    • Identify which components failed and why
    • Provide impact analysis on parent assembly

Phase 4: Hierarchical Integrity Reporting

User Action: Request verification report

System Responsibilities:

  1. Generate comprehensive hierarchical verification report:
    • Component breakdown with individual integrity status
    • Assembly verification results with dependency analysis
    • Verification timestamps for each component and assembly
    • Trust propagation analysis showing how component trust affects assembly
  2. Supply chain transparency:
    • Component provenance information
    • Manufacturer and supplier verification status
    • Version compatibility analysis
    • Substitution and upgrade recommendations
  3. Audit trail maintenance:
    • Store verification results at component and assembly levels
    • Maintain component dependency history
    • Enable verification proof sharing for component sub-assemblies
  4. Impact analysis tools:
    • Show which parent assemblies are affected by component changes
    • Provide upgrade and maintenance recommendations
    • Support compliance reporting for regulatory requirements

User Experience Requirements

UX1: Clear Feedback During Verification

The system shall provide users with real-time feedback during the verification process.

Requirements:

  • Progress indicator showing verification status
  • Clear messaging about which chunks are being verified
  • Immediate notification of any verification failures
  • Estimated time remaining for verification

UX2: Hierarchical Verification Options

The system shall offer users flexibility in verification approaches for complex resource assemblies.

Options:

  • Quick Verify: Verify only critical components and core assembly
  • Full Verify: Verify all components, chunks, and complete assembly hierarchy
  • Selective Verify: Verify specific components, files, or chunks at any level
  • Component Verify: Verify only selected components without full assembly
  • Background Verify: Run verification in background with progress notifications
  • Trust-Based Verify: Skip verification of pre-trusted components and assemblies
  • Supply Chain Verify: Include manufacturer and supplier verification in verification process

UX3: Hierarchical Error Handling and Recovery

The system shall handle verification failures gracefully at component and assembly levels.

Requirements:

  • Component-level error reporting: Clear error messages explaining which components failed verification
  • Assembly impact analysis: Show how component failures affect parent assemblies
  • Graceful degradation: Continue with available components while handling failed ones
  • Component substitution assistance: Suggest alternative components when available
  • Automatic retry mechanisms: Retry failed components and chunks with different sources
  • Recovery recommendations: Provide step-by-step guidance for resolving verification issues
  • Partial assembly support: Allow use of partially verified assemblies with clear warnings

Integration Requirements

I1: ValueFlows Compatibility

The system shall integrate seamlessly with existing ValueFlows resource management.

Requirements:

  • Integrity metadata stored with ResourceSpecification entries
  • Resource transfers include integrity verification steps
  • Economic events can depend on successful verification
  • Reputation systems consider verification success rates

I2: Governance Integration

The system shall respect existing governance rules and access controls.

Requirements:

  • Verification operations respect capability-based permissions
  • Audit trails for all verification activities
  • Resource owners can set verification requirements
  • Verification failures can trigger governance actions

I3: Standards Compliance

The system shall follow relevant industry standards for digital integrity.

Requirements:

  • Compatibility with IOPA (Internet of Production Alliance) standards
  • Support for common cryptographic standards (SHA-256, Merkle trees)
  • Documentation of verification processes for compliance audits
  • Extensible design for future standard updates

Performance Requirements

P1: Verification Speed

  • Individual chunk verification: <100ms per chunk
  • Full resource verification: <10 seconds for typical resources
  • Verification report generation: <1 second
  • Concurrent verification support for multiple resources

P2: Storage Efficiency

  • Manifest overhead: <1% of total resource size
  • Chunk size optimization for DHT performance
  • Efficient data structures for verification metadata
  • Minimal additional storage for verification data

P3: Network Efficiency

  • Parallel chunk retrieval for faster downloads
  • Optimized chunk location algorithms
  • Caching strategies for frequently accessed resources
  • Bandwidth-efficient verification protocols

Security Requirements

S1: Cryptographic Security

  • Use industry-standard cryptographic algorithms (SHA-256)
  • Proper random number generation for cryptographic operations
  • Protection against collision attacks through hash algorithms
  • Regular security reviews of cryptographic implementations

S2: Access Control

  • Verification operations respect existing access permissions
  • Audit trails for all integrity-related operations
  • Protection against unauthorized modification of verification data
  • Secure storage of cryptographic keys and certificates

S3: Data Privacy

  • Optional encryption for sensitive resource data
  • Privacy-preserving verification options when possible
  • Compliance with data protection regulations
  • User control over verification data sharing

Success Metrics

Technical Metrics

  • Verification success rate: >99.9%
  • False positive rate: <0.01%
  • Verification time within performance requirements
  • Resource availability with integrity guarantees

User Experience Metrics

  • User satisfaction with verification process
  • Trust levels in resource integrity system
  • Adoption rate of verification features
  • Support requests related to verification issues

Business Metrics

  • Reduced disputes over resource integrity
  • Increased trust in resource exchanges
  • Compliance with industry regulations
  • Competitive advantage through robust integrity guarantees

Technical Context (for Implementation Reference)

Storage Architecture

  • File Chunking: Resources split into 64KB chunks for DHT distribution
  • Content Addressing: Each chunk addressed by its SHA-256 hash
  • Manifest System: Resource manifests organize chunks and provide integrity metadata
  • Merkle Trees: Hierarchical hash structure enables efficient selective verification
  • Component References: Manifests can reference other resource manifests as components
  • Assembly Hierarchies: Support for unlimited nesting depth in resource compositions

Cryptographic Foundation

  • Hash Algorithm: SHA-256 for chunk and resource verification
  • Tree Structure: Merkle trees for efficient integrity proofs
  • Signature Scheme: Holochain's built-in cryptographic signatures
  • Validation: Multi-level verification (chunk → file → resource → component → assembly)
  • Trust Propagation: Cryptographic proof chaining through component hierarchies
  • Component Signatures: Individual component verification proofs that combine into assembly proofs

Composability Architecture

  • Atomic Resources: Base resources with no sub-components (files, data chunks)
  • Component Resources: Resources that can be used as parts of larger assemblies
  • Composite Resources: Assemblies built from multiple component resources
  • Assembly Rules: Constraints and requirements for valid component combinations
  • Version Matrices: Compatibility rules defining which component versions work together
  • Substitution Rules: Allowable component replacements with verification implications

Integration Points

  • ResourceSpecification: Enhanced with component references and assembly rules
  • Governance Zome: Access control and audit trail integration for component operations
  • Economic Events: Verification-dependent transaction flows for component exchanges
  • Reputation Systems: Agent reliability based on component and assembly verification history
  • Supply Chain Integration: External manufacturer and supplier verification systems
  • Standards Compliance: IOPA and other manufacturing standards for component assemblies

This requirements document provides the foundation for implementing a comprehensive digital resource integrity system that ensures trust and reliability in the nondominium resource exchange ecosystem.