Most large AI models are constrained by how much VRAM a single machine has. A model like Mixtral 8x22B requires roughly 140GB of VRAM — more than any single consumer GPU can hold. ZynkCluster proposes distributing that model across multiple devices on your local network, using each machine's GPU to run a different part of the model simultaneously. Models that are simply out of reach on a single consumer machine become accessible across hardware you already own.
No cloud. No subscription. No data leaving your network.
Today, anyone who wants to run AI privately faces a hard constraint: you're limited to whatever fits in a single GPU's memory. A typical PC with a gaming-grade graphics card — an NVIDIA RTX 3090, for example — has 24GB of VRAM. That's enough for a capable local model, but nowhere near the models that are genuinely impressive at reasoning, nuance, and long context. Those require 80GB, 140GB, more.
So in practice, people who want that level of quality send their data to a cloud service. That means your questions, your documents, your personal context — all of it going to a server you don't control, under terms of service you mostly didn't read, retained and used in ways that aren't publicly disclosed.
ZynkCluster's goal is to close that gap. Pool the GPU memory across two or three PCs on your home network, and models that were simply out of reach become runnable — privately, offline, at no ongoing cost.
Sensitive information you'd never type into ChatGPT. Medical questions, legal documents, financial records, therapy notes, business strategy. Anything where the answer to "where does this data go?" matters. With ZynkCluster, it doesn't go anywhere — it stays on your hardware.
Zynkbot with a capable model. Zynkbot already stores a detailed memory of who you are — your habits, your health, your family, your work. That's exactly the kind of data you don't want being used by a corporation to build a profile on you. Using a cloud API also means losing those capabilities when your internet is down, or if you're in a developing region or a heavily monitored network environment. ZynkCluster means Zynkbot can run a genuinely capable model against that memory without any of it leaving your network.
No content restrictions. Cloud providers impose policies on what their models will discuss — for liability reasons, business reasons, or both. Open models running locally have no such restrictions. For researchers, writers, medical professionals, anyone who desires a non-sanitized response or with other legitimate needs, that matters.
Hardware you already have, doing something useful. Most people with more than one computer have GPU capacity sitting idle. A gaming PC in another room, an older desktop that still works — pooled together over your home network, they become a meaningful compute cluster. You're not buying anything new; you're using what you own.
No ongoing cost. Download the model weights once. After that, inference is free. No subscription, no per-token billing, no API key.
Your phone as a cluster node (future). Modern smartphones have capable GPUs and dedicated neural processing units. Participating as a low-weight node — hosting one or two experts while plugged in and idle — is architecturally plausible and is a planned Phase 4 research area. See hardware notes below.
The current target is PCs with discrete GPUs running NVIDIA CUDA or AMD ROCm — the software layers that enable general-purpose GPU compute. Cards like the NVIDIA RTX 3060/3070/3080/3090 or AMD RX 6800/7900 series qualify. These are the same chips used in ML research; they're marketed and sold for gaming but are identical hardware.
Apple Silicon (M1/M2/M3/M4 Macs) — a near-term future target. Unified memory architecture means the GPU and CPU share the same RAM pool, so an M2 Max with 96GB can hold substantial model weight. Support via llama.cpp's Metal backend is architecturally plausible and on the roadmap.
Android phones — a longer-term research area. Modern flagship Android phones carry hardware that can run inference: Qualcomm Snapdragon chips include an Adreno GPU and a Hexagon NPU, and llama.cpp has a Qualcomm QNN backend. A phone with 12GB RAM could potentially host one or two experts from a smaller MoE model at 4-bit quantization. The real constraints are thermal throttling under sustained compute load, RAM shared with the OS, and battery drain. Practically, a phone would need to be plugged in, on WiFi, and idle. This is Phase 4 research — dependent on the core cluster proving out first.
Running LLMs on phones is already an active area of development. These projects demonstrate it's viable today on capable hardware, which is the foundation ZynkCluster would build on for mobile node participation:
Note: Ollama, LM Studio, and similar tools run on desktop only (Mac/Linux/Windows) — they do not run on mobile devices.
Gaming consoles (PlayStation, Xbox) — not viable. Sony and Microsoft lock their hardware completely; you cannot install compute drivers or run arbitrary workloads on them.
ZynkCluster is being developed by one person. There is no team, no funding, and no corporate roadmap behind it. That's worth being honest about — and it's also worth explaining why that's unlikely to change on its own.
The companies best positioned to build this — cloud providers, GPU manufacturers, AI labs — have little incentive to do so. Distributed consumer inference removes the cloud dependency that makes AI profitable. A tool that lets people run large models on hardware they already own, privately, on their own network, is structurally incompatible with a subscription or API billing model. The value flows entirely to users, not to a platform. That's exactly the point, and exactly why no well-funded team is racing to build it.
This is one of the things GitHub and open source exist for. If you have 2+ GPU-equipped machines and want to test the architecture, fork the repo and try it. If you're a developer interested in distributed inference, MoE routing, or network protocols and want to contribute, the code and documentation are there. If the proof of concept works, the implications are meaningful — not just for Zynkbot, but for anyone who wants to run large models privately on consumer hardware.
ZynkCluster is built around Mixture of Experts (MoE) models — architectures where each query activates only a small subset of the model's expert sub-networks. The number of active experts per token (k) is fixed by each model's training: Mixtral 8x7B and 8x22B use k=2, DBRX uses k=4, OLMoE uses k=8, and DeepSeek-V2 uses k=6. ZynkCluster's optimal cluster size matches the model's k — with a k=4 model, four devices run in parallel; with k=8, up to eight.
When a query needs Expert 2 and Expert 6, ZynkCluster routes them to two different devices and runs them at the same time. Both devices finish, the results are combined, and inference continues. The latency is determined by the slowest device — not the sum of all of them.
Petals (Sequential layers — all devices process every token): Device A → Device B → Device C → ... Total latency: sum of all device times + all network hops ZynkCluster (Parallel experts — only active devices used): Router selects Expert 2 + Expert 6 Device A (Expert 2) ◄── Parallel ──► Device B (Expert 6) Total latency: max(Device A, Device B) + 1 network hop
This is fundamentally different from existing distributed inference tools, which send every token through every device in sequence. ZynkCluster only involves the devices that are needed, and uses them simultaneously rather than one after another.
The device-pairing infrastructure built for ZynkSync — memory synchronization across your devices — becomes the coordination layer for compute clustering. The same discovery, authentication, and networking. Different payload.
The phrase "when a query needs Expert 2 and Expert 6" can sound like experts are predefined specialists with named jobs. They're not. Here's what's actually happening.
In an MoE model, each transformer layer contains several expert sub-networks — typically standard feed-forward networks with identical architecture but entirely independent weights. In Mixtral 8x7B, there are 8 experts per layer. No one assigned "Expert 2 handles grammar" or "Expert 6 handles code." The model developed its own internal division of labor through training, the same way any neural network learns. What each expert actually specializes in is largely opaque — researchers have found loose patterns, but nothing clean or interpretable. Think of them as 8 slightly different brains that the model learned to consult selectively.
Alongside the experts, each layer has a small router network (also called a gate). It takes the current hidden state — a vector representing the token being processed — and outputs a probability score for every expert. The top-k scores win. Those experts run; the rest don't.
Token hidden state → Router → score each expert: Expert: [ 0 1 2 3 4 5 6 7 ] Score: [ 0.05 0.31 0.42 0.08 0.03 0.07 0.38 0.12 ] Top-2 (k=2): Expert 2 (0.42) and Expert 6 (0.38) Experts 0, 1, 3, 4, 5, 7 → not activated for this token
"Expert 2 and Expert 6" is not a fixed choice. It changes with every single token, based on what the router scores for that specific hidden state. Different tokens in the same sentence often route to completely different expert pairs. The router itself is lightweight — roughly 100MB — and runs on the coordinator device.
Experts are logical units inside the model. Devices are physical hardware. ZynkCluster loads expert weights onto devices at startup and keeps them there. When the router fires, the coordinator dispatches to wherever the winning experts happen to live:
Startup (one-time weight loading):
Device A ← Experts 0, 1, 2, 3 (weights loaded into VRAM)
Device B ← Experts 4, 5, 6, 7 (weights loaded into VRAM)
Per-token dispatch (every token, dynamically):
Router selects Expert 2 and Expert 6 for this token
→ Expert 2 is on Device A → send hidden state to Device A
→ Expert 6 is on Device B → send hidden state to Device B
→ Both run simultaneously
→ Results return, coordinator combines them (weighted sum)
→ Next token
If router had selected Expert 2 and Expert 3 (both on Device A):
→ No parallelism benefit for that token — both run on Device A
The efficiency gain depends on the selected experts landing on different devices. Statistically, with well-distributed weights and random routing, this happens most of the time — but it's not guaranteed for every token. Tokens that route both experts to the same device pay no network overhead for that step.
The two tools I'm most aware of that attempt distributed LLM inference are Petals and DeepSpeed-MoE. There may be others. Neither of these targets consumer hardware the way ZynkCluster does, and the comparison below is theoretical — ZynkCluster has not been benchmarked against either.
| Metric | Petals (Sequential) | DeepSpeed-MoE | ZynkCluster |
|---|---|---|---|
| Distribution method | Layer by layer, sequential | Expert parallel | Expert parallel |
| Target hardware | Consumer (any) | Datacenter (InfiniBand) | Consumer (WiFi/Gigabit) |
| Active devices per token | All devices | Selected experts only | Selected experts only |
| Latency model | Sum of all hops | Max of parallel ops | Max of parallel ops |
| Privacy | Shared with cluster peers | Datacenter controlled | Local network only |
| Device pairing | Manual/public | Datacenter provisioning | ZynkSync (existing) |
DeepSpeed-MoE achieves expert-level parallelism but requires high-speed datacenter interconnects (InfiniBand) — not viable on consumer hardware. Petals works on consumer hardware but uses sequential layer routing, meaning every device in the cluster must process every token. ZynkCluster combines consumer hardware accessibility with expert-level parallelism.
The numbers below are theoretical estimates extrapolated from single-device benchmarks and known network overhead for hidden state tensor transfers. These have not been measured on a live cluster. They represent what the architecture should achieve if WiFi bandwidth and latency are within acceptable ranges — which is itself an open question. See the Developer Notes section below.
Model: Mixtral 8x7B (47B params, 13B active)
Tokens/sec: ~40
First token latency: ~50ms
Max model size: limited by single GPU VRAM
Model: Mixtral 8x22B (141B params, 39B active) — 3× larger
Tokens/sec: ~30–35 (estimated 75–87% of single device)
First token latency: ~75ms (estimated +25ms network)
Graceful degradation if one device drops
The projected result: two consumer GPUs should be able to run a significantly larger model than either could handle alone — the exact multiplier depends on the model's ratio of shared-to-expert parameters and how shared layers are handled across devices. Measuring the real-world capacity gain and latency cost on consumer hardware is one of the primary goals of this research.
ZynkCluster is theoretically sound and grounded in published research on Mixture of Experts architectures. The architecture has been reviewed and validated in principle. Development is ongoing; full implementation will proceed as other priorities — including Android support — are completed.
Illustrative architecture code, a visual flow diagram, and full documentation
are available in the repository under labs/moe_poc/.
If you're a developer interested in distributed inference and want to contribute,
that's the place to start.
ZynkCluster's theoretical foundation is solid — the MoE parallelism is real and inherent to how these models work, and the architecture follows logically from it. What hasn't been validated yet is how that theory holds up on consumer networking hardware. These are the open questions any contributor or tester should be aware of:
Passing hidden states between devices across a network layer requires serializing tensors and transmitting them over WiFi or Ethernet. The size of those tensors — and whether consumer bandwidth can handle them at inference speed — has not been measured. Gigabit Ethernet is likely fine; WiFi may introduce meaningful overhead depending on contention, distance, and interference.
The +25ms network overhead figure in the performance projections is an estimate based on typical LAN round-trip times plus rough tensor serialization overhead. It has not been measured on a live cluster. Real-world latency may be higher, particularly over WiFi or on congested networks.
Encoding and decoding tensors for network transfer adds CPU overhead that isn't present in single-device inference. The impact of this on throughput is unknown until tested.
The number of experts activated per token (k) is fixed by each model's training and cannot be changed without retraining. ZynkCluster's cluster size must match the chosen model's k. Testing should begin with a well-documented model like Mixtral 8x7B (k=2) before moving to higher-k models.
A successful proof of concept would demonstrate: expert routing over a local network, parallel execution on two or more GPU-equipped devices, result aggregation, and measured latency within acceptable range of the theoretical projections. Hardware requirements: 2+ devices with NVIDIA CUDA or AMD ROCm support, Gigabit network preferred.
ZynkCluster applies established MoE research to a deployment context that prior work hasn't targeted: consumer devices on local networks. The parallelism concept and MoE architecture are not new — the contribution is the deployment model.
ZynkCluster is not Mixtral-specific. Any open MoE model can be used; cluster size should match the model's k value.