Decentralized AI is regaining prominence in recent months as more people start to explore the intersection of Web3 and generative AI. While most people agree that decentralization can have positive benefits for AI, the specific technical paths run into major roadblocks.
For instance, while we all think that decentralized compute networks (DePIN) can help counterbalance the GPU accumulation games dominated by big tech platforms, the reality is that the pre-training and fine-tuning requirements of large foundation models require concentrated GPU topologies with massive communication buses.
Similarly, decentralized data networks seem like a clear idea to mitigate the increasing data concentration in large companies. But, in practice, the attempts to solve that problem have seen very mediocre traction.
When thinking about decentralizing AI, it is important to realize that AI tends to increasingly evolve more towards centralized architectures, which makes any decentralization effort incredibly challenging. For decentralized generative AI to overcome those natural challenges, it needs to rely on or influence four key trends:
- For decentralized AI to become feasible, open-source generative AI needs to become more mainstream
- Decentralized AI is an inference game and nothing else
- Web3 compute and data infrastructures need to scale to meet the demands of foundation mod
