FairScale Github

Table of Contents

stable

Index

What is FairScale?
User Workflow
Blogs and Press
Getting Involved
Integrations

Installation

Installing FairScale

Deep Dive

Efficient Memory management
OffloadModel
Adascale
Pipeline Parallelism
Enhanced Activation Checkpointing
SlowMo Distributed Data Parallel

Tutorials

Optimizer, Gradient and Model Sharding
Efficient memory usage using Activation Checkpointing
Scale your model on a single GPU using OffloadModel
Scale without modifying learning rate using Adascale
Model sharding using Pipeline Parallel
Tooling to diagnose and fix memory problems
Efficient Data Parallel Training with SlowMo Distributed Data Parallel

API Documentation

API Reference

Docs >
User Workflow
Edit on GitHub

Shortcuts

User Workflow¶

User workflow Diagram with explanation of various decision points

_images/flowchart.png

Next

Previous

© Copyright 2020-2022, Facebook/Meta AI Research. Revision 50b06d25.

Built with Sphinx using a theme provided by Read the Docs.

fairscale
Get Started
Contributing

Resources
Docs
Github Issues

To analyze traffic and optimize your experience, we serve cookies on this site. By clicking or navigating, you agree to allow our usage of cookies. As the current maintainers of this site, Facebook’s Cookies Policy applies. Learn more, including about available controls: Cookies Policy.

Get Started
Docs
Github