Pytest-xdist: Run tests in parallel

Problem: How do you run tests in parallel with pytest?

We have begun using pytest as our test runner at most of our clients. At one client, our tests are triggered by CircleCI and run on BrowserStack. Our GUI automation suite grew in size and was starting to take a long time to run against a vast combination of OSes and browser versions. Luckily, our client purchased 5 BrowserStack licenses for us. We wanted to run tests in parallel on the 5 channel BrowserStack from CircleCI using pytest. We searched on Google, but we didn’t get much information about running pytest in parallel. So this post is to help testers facing a similar issue. We will show you how to use pytest-xdist to run pytests in parallel.

Setup:

To run pytests in parallel, we need to install pytest-xdist plugin. To install x-dist plugin use:

pip install pytest-xdist

Command to run pytest in parallel:

The command to tell pytest to run your tests is parallel depends on the OS.

A)Windows:
To run tests in parallel on Windows, use:

py.test –n NUM

Here NUM = Number of parallel tests you want running at one time

In our case, we had 5 BrowserStack channels. So our command ended up being:

 py.test –n 5

B)Linux:

Our CircleCI container runs Ubuntu. To run tests in parallel from on Linux, use:

py.test -d --tx NUM*popen//python=python2.7

Here NUM = Number of parallel tests you want running at one time

In our case, our command ended up being:

py.test -d --tx 5*popen//python=python2.7

We liked pytest-xdist. It helped us distribute and run our tests in parallel in a straightforward way. We hope this helps you too.

If you liked this article, learn more about Qxf2’s testing services for startups.


Rohan Dudam

I love technology and learning new things. I explore both hardware and software. I am passionate about robotics and embedded systems which motivate me to develop my software and hardware skills. I have good knowledge of Python, Selenium, Arduino, C and hardware design. I have developed several robots and participated in robotics competitions. I am constantly exploring new test ideas and test tools for software and hardware. At Qxf2, I am working on developing hardware tools for automated tests ala Tapster. Incidentally, I created Qxf2’s first robot. Besides testing, I like playing cricket, badminton and developing embedded gadget for fun.

12 Comments

  1. Twirrim said:

    The -n argument works the same under Linux as it does under Windows. Is there a particular reason to take the longer form?

    May 30, 2017
    Reply
    • Shivahari P Shivahari P said:

      Hi,
      Thanks for pointing it out.We tried running tests using -n on Linux using Circle CI and it did not work hence the long form.

      June 2, 2017
      Reply
      • Anonymous said:

        Confirmed that the -n argument works in Linux.

        October 12, 2018
        Reply
        • Shivahari P Shivahari P said:

          Hi,
          Thanks for responding.
          What distribution of Linux did you try it against?

          October 15, 2018
          Reply
  2. Jalal kasmani said:

    How can we run all the test_.py files from the python script itself. I want to run all of my test_.py files by calling a function with a python script using tkinter . Can you kindly help advise.

    February 15, 2019
    Reply
    • Rohan Dudam Rohan Dudam said:

      Hi Jalal,
      To run all the test_.py files from python script, you need to add following lines in python script
      call_pytest_file.py
      import subprocess
      print subprocess.check_output(['pytest'])

      if you pytest command which you want to use contents spaces, you need follow below fashion
      #subprocess.check_output([‘ls’,’-l’])

      Hope it will help you
      thanks

      February 21, 2019
      Reply
  3. Anonymous said:

    While using xdist Im not able to call the pytest hook makereport

    March 26, 2019
    Reply
  4. Anonymous said:

    Can xdist run test parallel on multiple node instead of multiple browser?

    June 4, 2019
    Reply
  5. Anonymous said:

    If I have multiples tests that each take different command line arguments how do I use xdist to run them in parallel.

    Example:
    testsuite/
    test1.yaml
    test2.yaml
    test1.py
    test2.py

    If I were to run these tests serially the commands would look like
    python -m test1.py –config=test1.yaml
    python -m test2.py –config=test2.yaml

    January 9, 2020
    Reply
    • Nilaya Indurkar Nilaya Indurkar said:

      You can run the tests in parallel like below:
      pytest -n auto -m parallel test1.py –config=test1.yaml && pytest -n auto -m parallel test2.py –config=test2.yaml– html=report.html

      Check this too:
      https://github.com/pytest-dev/pytest-xdist/issues/325

      January 10, 2020
      Reply

Leave a Reply

Your email address will not be published.