斯坦福大学吴恩达教授最新来信：AI, GPU和芯片的未来

全球人工智能教育及研究领导者吴恩达教授最新来信：AI, GPU和芯片的未来

亲爱的朋友们：
近十年来，人工智能的兴起得益于GPU及其他加速器芯片速度的提高和成本的降低。这个趋势会持续多久？过去一个月发生了一些可能影响GPU价格变化的事件。

9月，支持以太加密货币的主要区块链以太坊（Ethereum）完成了一次转换，大大减少了所需的计算量。这种转变被称为“合并”（the Merge），它应该能够通过消耗更少的能源来造福自然环境。它还将减少对GPU进行加密货币挖掘的需求。（比特币区块链的计算成本仍然很高。）我预计较低的需求将有助于降低GPU价格。

另一方面，英伟达首席执行官黄仁勋近日宣布，预计芯片价格下跌的时代已经结束。他说，摩尔定律已经过时，这一长期趋势使大约每两年可以在硅的某一特定区域内安装的晶体管数量翻了一番。这一预测有多准确尚待观察。毕竟，早年间许多关于摩尔定律已经过时的报道都被证明是错误的。英特尔就笃定它会持续下去。

也就是说，GPU性能的改进已经超过了摩尔定律的速度，因为英伟达已经优化了其芯片以处理神经网络，而用于处理更大编程范围的CPU的改进速度已经落后。因此，即使芯片制造商无法用晶体管更密集地封装硅，芯片设计者也可以持续进行优化以提高AI的性价比。

尽管全球范围内对芯片的生产和需求出现波动，我对人工智能从业者将获得他们需要的处理能力依然持乐观态度。虽然使用更廉价的计算在更大的数据集上训练更大的神经网络来推动人工智能已取得了很大的进步，但现在其他创新引擎也在推动人工智能的发展。以数据为中心的人工智能、小数据、更高效的算法，以及正在进行的使人工智能适应数千（百万？）新应用程序的开发工作将推动事情向前发展。

近年来，半导体初创公司经历了一段艰难的时期，因为当他们赶上市场领导者英伟达提供的任何特定产品时，英伟达的重心却开始转向研发速度更快、更便宜的产品了。如果芯片价格停止下跌，他们将获得更大的市场机会来制造具有竞争力的芯片——尽管依然存在重大的技术壁垒。人工智能加速器行业仍然充满活力。英特尔和AMD正在进行重大投资，越来越多的公司正在MLPerf基准（用以衡量芯片性能）上进行较量。我相信在云端和边缘设备上进行训练和推理的选项将继续被扩展。

请不断学习！
吴恩达

英文原文：

Dear friends,
The rise of AI over the last decade has been powered by the increasing speed and decreasing cost of GPUs and other accelerator chips. How long will this continue? The past month saw several events that might affect how GPU prices evolve.In September, Ethereum, a major blockchain that supports the cryptocurrency known as ether, completed a shift that significantly reduced the computation it requires. This shift — dubbed the Merge — should benefit the natural environment by consuming less energy. It will also decrease demand for GPUs to carry out cryptocurrency mining. (The Bitcoin blockchain remains computationally expensive.) I expect that lower demand will help lower GPU prices.

On the other hand, Nvidia CEO Jensen Huang declared recently that the era in which chip prices could be expected to fall is over. Moore’s Law, the longstanding trend that has doubled the number of transistors that could fit in a given area of silicon roughly every two years, is dead, he said. It remains to be seen how accurate his prediction is. After all, many earlier reports of the death of Moore’s Law have turned out to be wrong. Intel continues to bet that it will hold up.

That said, improvements in GPU performance have exceeded the pace of Moore’s Law as Nvidia has optimized its chips to process neural networks, while the pace of improvements in CPUs, which are designed to process a wider range of programming, has fallen behind. So even if chip manufacturers can’t pack silicon more densely with transistors, chip designers may be able to continue optimizing to improve the price/performance ratio for AI.

I’m optimistic that AI practitioners will get the processing power they need. While much AI progress has been — and a meaningful fraction still is — driven by using cheaper computation to train bigger neural networks on bigger datasets, other engines of innovation now drive AI as well. Data-centric AI, small data, more efficient algorithms, and ongoing work to adapt AI to thousands (millions?) of new applications will keep things moving forward.Semiconductor startups have had a hard time in recent years because, by the time they caught up with any particular offering by market leader Nvidia, Nvidia had already moved on to a faster, cheaper product. If chip prices stop falling, they’ll have a bigger market opportunity — albeit with significant technical hurdles — to build competitive chips. The industry for AI accelerators remains dynamic. Intel and AMD are making significant investments and a growing number of companies are duking it out on the MLPerf benchmark that measures chip performance. I believe the options for training and inference in the cloud and at the edge will continue to expand.

Keep learning!
Andrew
转自知乎

从来信中看到吴教授的一些观点，大致可以总结如下：
1、GPU及其他加速器芯片速度的提高和成本的降低有助于人工智能的发展。
2、支持以太加密货币的主要区块链以太坊（Ethereum）完成转换，减少所需的计算量
3、对人工智能从业者将获得他们需要的处理能力依然持乐观态度

如何看待顶级大咖吴教授的来信，你有什么想法，欢迎留言。