Synapse
Synapse enables collaborative training of large AI models using distributed commodity hardware and spot cloud instances interconnected over the internet.
Key Capabilities
Parallel Training Modes
Supports data, model, mixture of experts, and hybrid parallelism strategies to optimize distributed model development
Flexible Hardware Options
Facilitates networks of CPUs and GPUs - on both commodity and industrial machines. Optimizes workload allocation.
Dynamic Network Topologies
Machines can volunteer or leave ongoing training as needed. Synapse auto load balances tasks.
Geo-distributed Pipelines
Leverage machines spread across geographic locations linked over the internet.
Outsized Model Handling
Modularly trains models too large for a single node's memory capacity alone.
Upcoming Features
Optimised Load Balancing
Improved load balancing and efficiency enhancements.
Security
Tighter integration with Syntience for security, incentives, and governance
FAQs
Understanding Synapse
What does Synapse do?
Synapse facilitates collaborative training of large neural network models using spare compute capacity volunteered from independent nodes.
What problems does it solve?
- Allows training giant models without massive centralized infrastructure
- Taps latent compute power left unused around the world
- Democratizes model development beyond well-funded entities
What can I use Synapse for?
You can leverage Synapse to:
- Modularly train models too large for a single commodity machine
- Harness geo-distributed resources, reducing need for centralized clusters
- Enable communities to voluntarily contribute capacity for shared innovations