talk · community record
Tensor and 2D Parallelism in PyTorch
DTensor and Tensor Parallel APIs in PyTorch Distributed, including Sequence Parallel and contributions to TCPStore and ProcessGroup for large training clusters.
DTensor and Tensor Parallel APIs in PyTorch Distributed, including Sequence Parallel and contributions to TCPStore and ProcessGroup for large training clusters.