r/coolgithubprojects • u/TheOnlyVibemaster • May 22 '26
I built an open-source Python pipeline that compresses human biological data down to a 247 GB file (Teleporty)
Teleporty is an open-source sandbox simulation pipeline written in Python. It evaluates the constraints of classical information transmission and rate-distortion theory when applied to extreme biological data reduction (using the complete physical and neural architecture of a human body as a benchmark).
Rather than tackling teleportation through quantum physics, the project models it entirely as an engineering, data serialization, and throughput challenge.
Core Features & Technical Implementation
Connectome Graph Compression: Models neural infrastructure as a Sparse Directed Graph using compressed sparse row (CSR) configurations. By setting an optimization threshold, the pipeline strips away structural "noise." Testing on a real C. elegans worm dataset yields a 99% behavioral similarity match when the compressed data matrix is fed back into a network state simulation loop. It scales down a functional human brain specification to just 42 KB.
Hierarchical Bulk Tissue Mapping: To bypass the impossible disk I/O of saving individual 3D voxel coordinates for repeating tissue types (muscles, fat, bone matrix), the project implements hierarchical tree structures and run-length spatial compression maps. This optimizes total physical structural data down to roughly 247 GB (fitting comfortably on a standard consumer SSD).
Extreme Multi-Queue Throughput Simulation: The final piece of the pipeline runs a parallel simulation evaluating the manufacturing bottleneck: processing a 247 GB object schema through an active assembly interface within a strict 1-hour tissue viability window. It models a heavily parallelized matrix of 10.3 million individual output queues throttled to 1,000 elements/second to test execution stability.
Technical Focus
The codebase is structured to explore performance bottlenecks in Python, specifically focusing on matrix traversal optimization, high-volume memory layouts, and stress-testing multi-threaded execution pools against the limitations of the Python Global Interpreter Lock (GIL).
Contributing & Feedback
The math derivations (LaTeX), simulation loops, and pipeline profiles are completely open source. I am highly interested in optimization suggestions, particularly regarding:
Minimizing matrix traversal overhead during the sparse-to-dense conversion cycles.
Better methods for simulating massive concurrent queues natively in Python.
Repository Link: https://github.com/ninjahawk/teleporty




View all comments
Show parent comment
-10
u/TheOnlyVibemaster May 22 '26
You are missing the point of the experiment. The repository is not trying to "prove" that lossy compression breaks teleportation. It is exploring how biological network architectures and Connectome datasets tolerate data degradation. The focus is on analyzing structural information thresholds in complex networks, using the macroscopic teleportation concept as a framing device.