Nextflow in Windows with WSL2

By Sarah Griffiths
Published in How Tos
October 20, 2021
3 min read

For EPI2ME Labs version 3 and onward the setup of Windows Subsytem for Linux is automated provided users have an up-to-date version of Windows 10 or 11. These instructions are provided to aid users on older versions of Windows that cannot update. For users who can update we (and Microsoft) recommend doing so before attempting to use the method presented here. The simple wsl --install method should be used in most cases, which is the method used in the automated setup within EPI2ME Labs 3 and newer. The methods here are not officially supported.

It is easy to run and develop Nextflow workflows on Windows but there are some additional set up steps required. We have found the set up can be tricky so have outlined the steps required here with links to trouble shooting and further help.

This guide is similar to that presented at a guide from Nextflow which goes in to greater detail and has additional steps useful for developing workflows.

You will need Windows 10 and it is critical that you have Admin privileges for all the steps,

  1. Check your windows version is sufficient

    • Start > search ‘Windows update settings’ > at the bottom select ‘OS Build info’
    • Under Windows specifications - Under OS Build info - Your Build number must be 18362.1049 or 18363.1049 (or higher).
    • Make any Windows updates that are required, this will cause your computer to restart and could take a while.
  1. Enable Windows Subsystem for Linux (WSL).

    If you have Windows 10 May 2020 (2004) update (or later) installed you can install WSL with a single command:

    • Start > search for ‘powershell’ > Right click and run as admin > copy/paste the following -
    wsl.exe --install
    • Restart your computer and you will be prompted to set up Ubuntu with a username and password and can skip to step 8.

    If you have an earlier Windows version or the above has not worked continue following the steps -

    • Start > search for ‘powershell’ > Right click and run as admin > copy/paste the following -
    dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart
    • Copy and paste the following in to Powershell
    dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart
  2. Update to WSL2

    • Download this WSL2 update file and run through the installation wizard: wsl_update.msi

    • A guide to WSL installation troubleshooting can be found here.

  3. Set WSL2 as the default version.

    • Open a new Powershell with admin rights
    • In Powershell copy and paste
    wsl --set-default-version 2
  4. Download WSL Ubuntu

    • From the start menu open the Microsoft store
    • In search bar type Ubuntu
    • Download and install the Ubuntu 64-bit version: Focal 20.04 (LTS).
  5. Check WSL version for Ubuntu installation

    WSL installations come in two versions, creatively name 1 and 2. We must ensure the Ubuntu installation is set to version 2.

    • In Powershell run:

      wsl --list –v

      The response should look something like:

      H:\> wsl --list -v
      NAME STATE VERSION
      * Ubuntu Running 2
      docker-desktop-data Running 2
      docker-desktop Running 2
      Ubuntu-18.04 Running 1
    • If a version is listed as 1 and not 2, change the version with

      wsl --set-version <distribution name> <version>

      For example to change the Ubuntu-18.04 installation above to version 2:

      H:\> wsl --set-version Ubuntu 2
  6. Set up Docker

    • Download Docker Desktop
    • Run the installation as an administrator and follow the install wizard.
    • Add your computer username to the “docker group” in WSL - in Powershell type:
    wsl
    sudo groupadd docker
    sudo usermod -aG docker $(whoami)
    • Reboot your PC
  7. Enable WSL2 in docker

    These docker instructions can also be found here

    • Start Docker Desktop from the Windows Start menu.
    • From the Docker menu, select Settings > General
    • The ‘Use the WSL 2 based engine’ box should already be ticked, if not tick the box and click Apply & Restart.

    Docker settings
    Docker settings

    • Go to Settings > Resources > WSL Integration
    • ‘Enable integration with my default WSL distro’ should be ticked and select Enable integration with the Ubuntu version you have just downloaded.

    Docker WSL Integration
    Docker WSL Integration

    • If you are unsure of the Ubuntu version go to Powershell and type
    wsl
    lsb_release -d

    This will respond with something like:

    Description: Ubuntu 20.04.3 LT0
  8. Set WSL2 resource limits

    It is important to configure docker resource limits so it does not use all available memory and cpu.

    • You will need to update a different file depending on if you want a global configurartion for all WSL distributions or a specific one for each distribution.

      • Global

        • In Windows Explorer navigate to C:/Users/<USERNAME>
        • Create or open existing file .wslconfig
      • Specific Distribution

        • Open specific distribution terminal eg. Ubuntu or type wsl Powershell
        • Navigate to the /etc and open a file:
        cd /etc
        sudo vim wsl.conf
      • Copy and paste the following in to the respective file

        [wsl2]
        memory=8GB
        processors=4
        localhostForwarding=true
    • Adjust settings in the config based on your available resources. To find out total available on your device you can CTRL-ALT-DELETE to open task manager, click performance tab and look at details of CPU and memory. Total no. of processors will be ‘logical processors’. Remember to leave some resource available for other programmes on your computer.

    • Save the file (in vim press Esc and type :wq!).

    • Shut down all running wsl instances via cmd line

      wsl --shutdown
    • Now relaunch wsl and check resources are as expected using cmds

      cat /proc/cpuinfo
      cat /proc/meminfo

    For more advanced settings see here

  1. Install Java

    To install Java into the WSL2 Ubuntu installation run (after starting WSL with the wsl command):

    sudo apt install openjdk-11-jre-headless
  2. Download and install Nextflow

    Nextflow can be installed following the instruction in their Getting started pages:

    • Run the following:
    curl -s https://get.nextflow.io | bash
    mv nextflow /usr/bin/nextflow
    chmod +x /usr/bin/nextflow
    • Make sure Nextflow is updated to the latest version by typing -
    nextflow self-update
  3. Test nextflow

    To test Nextflow we can run a simple workflow template.

    • To obtain some test data, download the workflow:
    git clone https://github.com/epi2me-labs/wf-template.git
    • To see required parameters of the workflow run:
    nextflow run epi2me-labs/wf-template
    • And to run the workflow with the test data run:
    OUTPUT=output
    nextflow run epi2me-labs/wf-template \
    -w ${OUTPUT}/workspace \
    --fastq wf-template/test_data --out_dir wf-template/${OUTPUT}

If you had any problems with this tutorial or think any additional information would be useful then please let us know so we can improve it for other users.


Tags

#workflows#wsl2#nextflow#windows

Share

Sarah Griffiths

Sarah Griffiths

Bioinformatician

Related Posts

Unexpected results, so now what?
July 02, 2024
3 min

Quick Links

TutorialsWorkflowsOpen DataContact

Social Media

© 2020 - 2024 Oxford Nanopore Technologies plc. All rights reserved. Registered Office: Gosling Building, Edmund Halley Road, Oxford Science Park, OX4 4DQ, UK | Registered No. 05386273 | VAT No 336942382. Oxford Nanopore Technologies, the Wheel icon, EPI2ME, Flongle, GridION, Metrichor, MinION, MinIT, MinKNOW, Plongle, PromethION, SmidgION, Ubik and VolTRAX are registered trademarks of Oxford Nanopore Technologies plc in various countries. Oxford Nanopore Technologies products are not intended for use for health assessment or to diagnose, treat, mitigate, cure, or prevent any disease or condition.