Installation#
Note
There are several ways to install Modin. Most users will want to install with
pip
or using conda
tool, but some users may want to build from the master branch
on the GitHub repo. The master branch has the most recent patches, but may be less
stable than a release installed from pip
or conda
.
Installing with pip#
Stable version#
Modin can be installed with pip
on Linux, Windows and MacOS.
To install the most recent stable release run the following:
pip install -U modin # -U for upgrade in case you have an older version
Modin can be used with Ray, Dask, Unidist, or HDK engines. If you don’t have Ray, Dask or Unidist installed, you will need to install Modin with one of the targets:
pip install "modin[ray]" # Install Modin dependencies and Ray to run on Ray
pip install "modin[dask]" # Install Modin dependencies and Dask to run on Dask
pip install "modin[unidist]" # Install Modin dependencies and Unidist to run on Unidist
pip install "modin[all]" # Install all of the above
Modin will automatically detect which engine you have installed and use that for scheduling computation! See below for HDK engine installation.
Release candidates#
Before most major releases, we will upload a release candidate to test and check if there are any problems. If you would like to install a pre-release of Modin, run the following:
pip install --pre modin
These pre-releases are uploaded for dependencies and users to test their existing code to ensure that it still works. If you find something wrong, please raise an issue or email the bug reporter: bug_reports@modin.org.
Installing specific dependency sets#
Modin has a number of specific dependency sets for running Modin on different execution engines and storage formats or for different functionalities of Modin. Here is a list of dependency sets for Modin:
pip install "modin[ray]" # If you want to use the Ray execution engine
pip install "modin[dask]" # If you want to use the Dask execution engine
pip install "modin[unidist]" # If you want to use the Unidist execution engine
Installing on Google Colab#
Modin can be used with Google Colab via the pip
command, by running the following code in a new cell:
!pip install "modin[all]"
Since Colab preloads several of Modin’s dependencies by default, we need to restart the Colab environment once Modin is installed by either clicking on the "RESTART RUNTIME"
button in the installation output or by run the following code:
# Post-install automatically kill and restart Colab environment
import os
os.kill(os.getpid(), 9)
Once you have restarted the Colab environment, you can use Modin in Colab in subsequent sessions.
Note that on the free version of Colab, there is a limit on the compute resource. To leverage the full power of Modin, you may have to upgrade to Colab Pro to get access to more compute resources.
Installing with conda#
Using conda-forge channel#
Modin releases can be installed using conda
from conda-forge channel. Starting from 0.10.1
it is possible to install modin with chosen engine(s) alongside. Current options are:
Package name in conda-forge |
Engine(s) |
Supported OSs |
modin |
Linux, Windows, MacOS |
|
modin-dask |
Dask |
Linux, Windows, MacOS |
modin-ray |
Linux, Windows |
|
modin-unidist |
Linux, Windows, MacOS |
|
modin-hdk |
Linux |
|
modin-all |
Dask, Ray, Unidist, HDK |
Linux |
For installing Dask, Ray and Unidist engines into conda environment following command should be used:
conda install -c conda-forge modin-ray modin-dask modin-unidist
All set of engines could be available in conda environment by specifying:
conda install -c conda-forge modin-all
or explicitly:
conda install -c conda-forge modin-ray modin-dask modin-unidist modin-hdk
conda
may be slow installing modin-all
or combitations of execution engines so we currently recommend using libmamba solver for the installation process.
To do this install it in a base environment:
conda install -n base conda-libmamba-solver
Then it can be used during installation either like
conda install -c conda-forge modin-ray modin-hdk --experimental-solver=libmamba
or starting from conda 22.11 and libmamba solver 22.12 versions
conda install -c conda-forge modin-ray modin-hdk --solver=libmamba
Using Intel® Distribution of Modin#
With conda
it is also possible to install Intel Distribution of Modin, a special version of Modin
that is part of Intel® oneAPI AI Analytics Toolkit. This version of Modin is powered by HDK
engine that contains a bunch of optimizations for Intel hardware. More details to get started can be found in the Intel Distribution of Modin Getting Started guide.
Installing from the GitHub master branch#
If you’d like to try Modin using the most recent updates from the master branch, you can
also use pip
.
pip install "modin[all] @ git+https://github.com/modin-project/modin"
This will install directly from the repo without you having to manually clone it! Please be aware that these changes have not made it into a release and may not be completely stable.
If you would like to install Modin with a specific engine, you can use modin[ray]
or modin[dask]
or modin[unidist]
instead of modin[all]
in the command above.
Windows#
All Modin engines except HDK are available both on Windows and Linux as mentioned above. Default engine on Windows is Ray. It is also possible to use Windows Subsystem For Linux (WSL), but this is generally not recommended due to the limitations and poor performance of Ray on WSL, a roughly 2-3x worse than native Windows.
Building Modin from Source#
If you’re planning on contributing to Modin, you will need to ensure that you are building Modin from the local repository that you are working off of. Occasionally, there are issues in overlapping Modin installs from pypi and from source. To avoid these issues, we recommend uninstalling Modin before you install from source:
pip uninstall modin
To build from source, you first must clone the repo. We recommend forking the repository first through the GitHub interface, then cloning as follows:
git clone https://github.com/<your-github-username>/modin.git
Once cloned, cd
into the modin
directory and use pip
to install:
cd modin
pip install -e .
pip install -e ".[all]" # will install dependencies for all engines