Pytest-xdist: Run tests in parallel

Problem: How do you run tests in parallel with pytest?

We have begun using pytest as our test runner at most of our clients. At one client, our tests are triggered by CircleCI and run on BrowserStack. Our GUI automation suite grew in size and was starting to take a long time to run against a vast combination of OSes and browser versions. Luckily, our client purchased 5 BrowserStack licenses for us. We wanted to run tests in parallel on the 5 channel BrowserStack from CircleCI using pytest. We searched on Google, but we didn’t get much information about running pytest in parallel. So this post is to help testers facing a similar issue. We will show you how to use pytest-xdist to run pytests in parallel.

Setup:

To run pytests in parallel, we need to install pytest-xdist plugin. To install x-dist plugin use:

pip install pytest-xdist

Command to run pytest in parallel:

The command to tell pytest to run your tests is parallel depends on the OS.

A)Windows:
To run tests in parallel on Windows, use:

py.test –n NUM

Here NUM = Number of parallel tests you want running at one time

In our case, we had 5 BrowserStack channels. So our command ended up being:

 py.test –n 5

B)Linux:

Our CircleCI container runs Ubuntu. To run tests in parallel from on Linux, use:

py.test -d --tx NUM*popen//python=python2.7

Here NUM = Number of parallel tests you want running at one time

In our case, our command ended up being:

py.test -d --tx 5*popen//python=python2.7

We liked pytest-xdist. It helped us distribute and run our tests in parallel in a straightforward way. We hope this helps you too.

If you liked this article, learn more about Qxf2’s testing services for startups.


12 thoughts on “Pytest-xdist: Run tests in parallel

  1. The -n argument works the same under Linux as it does under Windows. Is there a particular reason to take the longer form?

    1. Hi,
      Thanks for pointing it out.We tried running tests using -n on Linux using Circle CI and it did not work hence the long form.

  2. How can we run all the test_.py files from the python script itself. I want to run all of my test_.py files by calling a function with a python script using tkinter . Can you kindly help advise.

    1. Hi Jalal,
      To run all the test_.py files from python script, you need to add following lines in python script
      call_pytest_file.py
      import subprocess
      print subprocess.check_output(['pytest'])

      if you pytest command which you want to use contents spaces, you need follow below fashion
      #subprocess.check_output([‘ls’,’-l’])

      Hope it will help you
      thanks

  3. If I have multiples tests that each take different command line arguments how do I use xdist to run them in parallel.

    Example:
    testsuite/
    test1.yaml
    test2.yaml
    test1.py
    test2.py

    If I were to run these tests serially the commands would look like
    python -m test1.py –config=test1.yaml
    python -m test2.py –config=test2.yaml

Leave a Reply

Your email address will not be published.