SSH using Python Paramiko

Recently, as part of an automated test, we needed to SSH into a server, toggle a service and then check the response on a web application. We used the Python module Paramiko. We ended up writing simple wrappers around the most common actions test automation may need to perform over SSH. In this post, we will share the same.

We have written this tutorial for absolute beginners. If you are already familiar with SSH (there are people who are not!) and comfortable with Python, please jump ahead to the section ‘Putting it all together’.

Note 1: We have also integrated this code into our open-sourced Python test automation framework.
Note 2: The Python 3 code for this article is here

How SSH Works

The SSH connection is implemented using a client-server model. To establish a connection the server should be running and clients generally authenticated either using passwords or SSH keys. Password Authentication is simple and straightforward. To authenticate using SSH keys, a user must have an SSH key pair (Public and Private key). On the remote server, the list of public keys is maintained (usually) in the ~/.ssh/authorized_keys directory. When the client connects to the remote server using the public key, the server checks for it and sends an encrypted message which can only be decrypted with the associated private key at the client side. We will be using a Python module called Paramiko. The Paramiko module gives an abstraction of the SSHv2 protocol with both the client side and server side functionality. As a client, you can authenticate yourself using a password or key and as a server, you can decide which users are allowed access and the channels you allow.

Installing Paramiko

Installing Paramiko is straightforward, it has only one direct hard dependency: the Cryptography library. If pip is above 8.x, nothing else is required. pip will install statically compiled binary archives of Cryptography its dependencies. For more details on installation, please refer to this Paramiko’s documentation.

To install Paramiko on Windows

pip install paramiko

Overview of the Parmiko tutorial

In the coming sections, we will be talking about below items:

  • Connecting to the remote server using Paramiko
  • Executing commands on the server
  • File transfers
  • Putting it all together

    • We have written different methods for each functionality. All the parameters are read from a configuration file. We are reading the parameters like credentials, commands & file paths from the conf file because when there is any change in the machine or any credential changes, changing the configuration file alone is enough instead of changing the script.

    After importing Paramiko, to make everything more modular, we created a class called Ssh_Util and wrote different methods for each functionality.

    Connecting to the remote server using Paramiko

    Firstly, we created an initialization function inside the class and initialized required variables. Some of the parameters like host, username, password etc., are read from a configuration file.

    def __init__(self):
        self.ssh_output = None
        self.ssh_error = None
        self.client = None
        self.host= conf_file.HOST
        self.username = conf_file.USERNAME
        self.password = conf_file.PASSWORD
        self.timeout = float(conf_file.TIMEOUT)
        self.commands = conf_file.COMMANDS
        self.pkey = conf_file.PKEY
        self.port = conf_file.PORT
        self.uploadremotefilepath = conf_file.UPLOADREMOTEFILEPATH
        self.uploadlocalfilepath = conf_file.UPLOADLOCALFILEPATH
        self.downloadremotefilepath = conf_file.DOWNLOADREMOTEFILEPATH
        self.downloadlocalfilepath = conf_file.DOWNLOADLOCALFILEPATH

    We created a function connect() and placed inside the Ssh_Util class. This function will be used to connect to the remote server.

    def connect(self):
        "Login to the remote server"
        try:
            #Paramiko.SSHClient can be used to make connections to the remote server and transfer files
            print("Establishing ssh connection")
            self.client = paramiko.SSHClient()
            #Parsing an instance of the AutoAddPolicy to set_missing_host_key_policy() changes it to allow any host.
            self.client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
            #Connect to the server
            if (self.password == ''):
                private_key = paramiko.RSAKey.from_private_key_file(self.pkey)
                self.client.connect(hostname=self.host, port=self.port, username=self.username, pkey=private_key, timeout=self.timeout, allow_agent=False,look_for_keys=False)
                print("Connected to the server",self.host)
            else:
                self.client.connect(hostname=self.host, port=self.port, username=self.username, password=self.password,timeout=self.timeout, allow_agent=False,look_for_keys=False)    
                print("Connected to the server",self.host)
        except paramiko.AuthenticationException:
            print("Authentication failed, please verify your credentials")
            result_flag = False
        except paramiko.SSHException as sshException:
            print("Could not establish SSH connection: %s" % sshException)
            result_flag = False
        except socket.timeout as e:
            print("Connection timed out")
            result_flag = False
        except Exception,e:
            print("Exception in connecting to the server")
            print("PYTHON SAYS:",e)
            result_flag = False
            self.client.close()
        else:
            result_flag = True
 
        return result_flag

    Firstly, we created an instance ‘client’ of paramiko.SSHClient(). Paramiko.SSHClient is the primary client used to make connections to the remote server and execute commands. This class wraps Transport, Channel, and SFTPClient to take care of most aspects of authenticating and opening channels. The creation of an SSHClient object allows establishing server connections via the connect() method.

    When we try to connect to a server for the first time, we see an error message prompted saying that the machine is not informed about the remote server that we are trying to access. Paramiko’s client.set_missing_host_key_policy is the method used for this purpose. By default, the paramiko.SSHclient sets the policy to the Reject policy which means the policy rejects connection without validating. In our code, we are overriding this by passing AutoAddPolicy() function where the new server will automatically add the server’s host key without prompting. Note:- In a testing environment, we can use set_missing_host_key_policy and set AutoAddPolicy but for security purpose, this is not a good idea to use in production.

    Paramiko supports both password-based authentication and key-pair based authentication for accessing the server. In our code, we are checking for a password and if available, authentication is attempted using plain username/password authentication. If the password is not available, authentication is attempted by reading the private key file.

    client.Connect() is to connect to an SSH server. This method also allows to provide our own private key, or connect to the SSH agent on the local machine or read from the user’s local key files. We also used timeout (in seconds) to wait for an authentication response. We are capturing a couple of exceptions like paramiko.AuthenticationException, paramiko.SSHException, Socket.timeout to handle the errors while connecting.

    Executing commands on the server

    Now, you are connected to the remote server. The next step is to execute commands on the SSH server. To run a command on the server the exec_command() function is called on the SSHClient with the command passed as input. When you execute commands using exec_command a new Channel is opened and the requested command is executed. The response is returned as Python file-like objects representing stdin, stdout, and stderr(as a 3-tuple)

    • The stdin is a write-only file which can be used for input commands.
    • The stdout file give the output of the command.
    • The stderr gives the errors returned on executing the command. Will be empty if there is no error.

    We have written below function execute_command() to execute the commands. The input for this function is the set of commands. The call to the function connect() is made first and once the connection is established it is followed by a call to function exec_command(). We are storing the stdout and stderr into two variables namely ssh_output and ssh_error respectively. The ssh_output is the output to standard output as produced when the command executed on the remote computer. The ssh_error is the output of standard error produced at the same occasion. The user can use these variables for further processing if needed.

    def execute_command(self,commands):
        """Execute a command on the remote host.Return a tuple containing
        an integer status and a two strings, the first containing stdout
        and the second containing stderr from the command."""
        self.ssh_output = None
        result_flag = True
        try:
            if self.connect():
                for command in commands:
                    print("Executing command --> {}".format(command))
                    stdin, stdout, stderr = self.client.exec_command(command,timeout=10)
                    self.ssh_output = stdout.read()
                    self.ssh_error = stderr.read()
                    if self.ssh_error:
                        print("Problem occurred while running command:"+ command + " The error is " + self.ssh_error)
                        result_flag = False
                    else:    
                        print("Command execution completed successfully",command)
                    self.client.close()
            else:
                print("Could not establish SSH connection")
                result_flag = False   
        except socket.timeout as e:
            print("Command timed out.", command)
            self.client.close()
            result_flag = False                
        except paramiko.SSHException:
            print("Failed to execute the command!",command)
            self.client.close()
            result_flag = False    
 
        return result_flag

    File Transfers

    File transfers are needed to perform remote file operations. Paramiko allows to programmatically send and receive files using the SFTP protocol, the connection with the remote host is established in the same way explained in the previous sections, the call to connect() is followed by a call to open_sftp() that returns a new SFTPClient session object. This object allows to perform common SFTP operations like get(), put(),listdir().

    We have created two functions(upload_file() and download_file()) to upload a file to the remote server and download a file from the remote server.

    Firstly, we created a function upload_file() which uploads a file to the remote server. The code below establishes the SFTP Connection using the SSH client and uploads a file. The put() method will copy a local file (local path) to the SFTP server as the remote path. Note:- The filename should be included. Only specifying a directory may result in an error. Once the operation is done you may close the SFTP session and its underlying channel using ftp_client.close().

    def upload_file(self,uploadlocalfilepath,uploadremotefilepath):
        "This method uploads the file to remote server"
        result_flag = True
        try:
            if self.connect():
                ftp_client= self.client.open_sftp()
                ftp_client.put(uploadlocalfilepath,uploadremotefilepath)
                ftp_client.close() 
                self.client.close()
            else:
                print("Could not establish SSH connection")
                result_flag = False  
        except Exception,e:
            print('\nUnable to upload the file to the remote server',uploadremotefilepath)
            print('PYTHON SAYS:',e)
            result_flag = False
            ftp_client.close()
            self.client.close()
 
        return result_flag

    Similarly, download_file() function downloads a file from the remote server. The code below establishes the SFTP Connection using the SSH client and downloads a file. The get() method will copy a remote file (remote path) from the SFTP server to the local host as local path. Once the operation is done you may close the SFTP session and its underlying channel using ftp_client.close().

    def download_file(self,downloadremotefilepath,downloadlocalfilepath):
        "This method downloads the file from remote server"
        result_flag = True
        try:
            if self.connect():
                ftp_client= self.client.open_sftp()
                ftp_client.get(downloadremotefilepath,downloadlocalfilepath)
                ftp_client.close()  
                self.client.close()
            else:
                print("Could not establish SSH connection")
                result_flag = False  
        except Exception,e:
            print('\nUnable to download the file from the remote server',downloadremotefilepath)
            print('PYTHON SAYS:',e)
            result_flag = False
            ftp_client.close()
            self.client.close()
 
        return result_flag

    Putting it all together

    Note: The Python 3 code for this article is here. Please use it instead of the file below.

    import paramiko
import os,sys,time
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from conf import ssh_conf as conf_file   
import socket
 
class Ssh_Util:
    "Class to connect to remote server" 
 
    def __init__(self):
        self.ssh_output = None
        self.ssh_error = None
        self.client = None
        self.host= conf_file.HOST
        self.username = conf_file.USERNAME
        self.password = conf_file.PASSWORD
        self.timeout = float(conf_file.TIMEOUT)
        self.commands = conf_file.COMMANDS
        self.pkey = conf_file.PKEY
        self.port = conf_file.PORT
        self.uploadremotefilepath = conf_file.UPLOADREMOTEFILEPATH
        self.uploadlocalfilepath = conf_file.UPLOADLOCALFILEPATH
        self.downloadremotefilepath = conf_file.DOWNLOADREMOTEFILEPATH
        self.downloadlocalfilepath = conf_file.DOWNLOADLOCALFILEPATH
 
    def connect(self):
        "Login to the remote server"
        try:
            #Paramiko.SSHClient can be used to make connections to the remote server and transfer files
            print("Establishing ssh connection")
            self.client = paramiko.SSHClient()
            #Parsing an instance of the AutoAddPolicy to set_missing_host_key_policy() changes it to allow any host.
            self.client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
            #Connect to the server
            if (self.password == ''):
                self.pkey = paramiko.RSAKey.from_private_key_file(self.pkey)
                self.client.connect(hostname=self.host, port=self.port, username=self.username,pkey=self.pkey ,timeout=self.timeout, allow_agent=False, look_for_keys=False)
                print("Connected to the server",self.host)
            else:
                self.client.connect(hostname=self.host, port=self.port,username=self.username,password=self.password,timeout=self.timeout, allow_agent=False, look_for_keys=False)    
                print("Connected to the server",self.host)
        except paramiko.AuthenticationException:
            print("Authentication failed, please verify your credentials")
            result_flag = False
        except paramiko.SSHException as sshException:
            print("Could not establish SSH connection: %s" % sshException)
            result_flag = False
        except socket.timeout as e:
            print("Connection timed out")
            result_flag = False
        except Exception,e:
            print('\nException in connecting to the server')
            print('PYTHON SAYS:',e)
            result_flag = False
            self.client.close()
        else:
            result_flag = True
 
        return result_flag    
 
    def execute_command(self,commands):
        """Execute a command on the remote host.Return a tuple containing
        an integer status and a two strings, the first containing stdout
        and the second containing stderr from the command."""
        self.ssh_output = None
        result_flag = True
        try:
            if self.connect():
                for command in commands:
                    print("Executing command --> {}".format(command))
                    stdin, stdout, stderr = self.client.exec_command(command,timeout=10)
                    self.ssh_output = stdout.read()
                    self.ssh_error = stderr.read()
                    if self.ssh_error:
                        print("Problem occurred while running command:"+ command + " The error is " + self.ssh_error)
                        result_flag = False
                    else:    
                        print("Command execution completed successfully",command)
                    self.client.close()
            else:
                print("Could not establish SSH connection")
                result_flag = False   
        except socket.timeout as e:
            print("Command timed out.", command)
            self.client.close()
            result_flag = False                
        except paramiko.SSHException:
            print("Failed to execute the command!",command)
            self.client.close()
            result_flag = False    
 
        return result_flag
 
    def upload_file(self,uploadlocalfilepath,uploadremotefilepath):
        "This method uploads the file to remote server"
        result_flag = True
        try:
            if self.connect():
                ftp_client= self.client.open_sftp()
                ftp_client.put(uploadlocalfilepath,uploadremotefilepath)
                ftp_client.close() 
                self.client.close()
            else:
                print("Could not establish SSH connection")
                result_flag = False  
        except Exception,e:
            print('\nUnable to upload the file to the remote server',uploadremotefilepath)
            print('PYTHON SAYS:',e)
            result_flag = False
            ftp_client.close()
            self.client.close()
 
        return result_flag
 
    def download_file(self,downloadremotefilepath,downloadlocalfilepath):
        "This method downloads the file from remote server"
        result_flag = True
        try:
            if self.connect():
                ftp_client= self.client.open_sftp()
                ftp_client.get(downloadremotefilepath,downloadlocalfilepath)
                ftp_client.close()  
                self.client.close()
            else:
                print("Could not establish SSH connection")
                result_flag = False  
        except Exception,e:
            print('\nUnable to download the file from the remote server',downloadremotefilepath)
            print('PYTHON SAYS:',e)
            result_flag = False
            ftp_client.close()
            self.client.close()
 
        return result_flag
 
 
#---USAGE EXAMPLES
if __name__=='__main__':
    print("Start of %s"%__file__)
 
    #Initialize the ssh object
    ssh_obj = Ssh_Util()
 
    #Sample code to execute commands
    if ssh_obj.execute_command(ssh_obj.commands) is True:
        print("Commands executed successfully\n")
    else:
        print("Unable to execute the commands")
 
    """
    #Sample code to upload a file to the server
    if ssh_obj.upload_file(ssh_obj.uploadlocalfilepath,ssh_obj.uploadremotefilepath) is True:
        print("File uploaded successfully", ssh_obj.uploadremotefilepath)
    else:
        print("Failed to upload the file")
 
    #Sample code to download a file from the server
    if ssh_obj.download_file(ssh_obj.downloadremotefilepath,ssh_obj.downloadlocalfilepath) is True:
        print("File downloaded successfully", ssh_obj.downloadlocalfilepath)
    else:
        print("Failed to download the file")
    """

    Hope this code helps you get started with Paramiko easily. Happy coding! If you liked this article and want to learn more about Qxf2 and our testing services for startups, click here.

    References

    1) SSH and transfer file using paramiko
    2) How to SSH in Python using paramiko
    3) Paramiko Github examples
    4) How does SSH works?

    Related posts:

    Default ThumbnailGetting started with MongoDB and Python Default ThumbnailGet started with SQLite and Python Default ThumbnailReporting automation results to TestRail using Python
    Indira Nellutla

    64 thoughts on “SSH using Python Paramiko”

    1. The script background with teh alternating grey and white lines makes it unreadable. My eyes are hurting.

      Why not give it a mono color background like in any IDE. The code coloring is good but the zebra background destroys it. At least on my monitor

      Reply

    3. Hi,
      Iam trying to do an ssh to a server using paramiko . The below is my server structure where after logging in, again I need to do a “bwcli” where I will be running some commnads.
      Iam able to login, run commands like “ls”, “pwd” ,ect but when I run the “bwcli” , it just does not work.
      Is bwcli, creating another shell ?? Iam not able to figure the issue. Any help would be appreciated.
      ===============================================================
      bwadmin@MBCV-AS$ bwcli
      ======================================================================
      BroadWorks Command Line Interface
      Type HELP for more information
      ======================================================================
      Reading initial CLI command file…

      AS_CLI>
      =======================================
      My code :
      ssh = paramiko.SSHClient()
      ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
      ssh.connect(hostname = “x.x.x.x”, port =22, username = “user”, password = “pass”)
      ssh.invoke_shell()
      stdin,stdout,stderr=ssh.exec_command(“ls”)
      output = stdout.readlines()
      print (‘\n’.join(output))
      ssh.close()

      Reply

      1. Hi Sunil,
        I am not sure what bwcli does, but looks like bwcli opens up the interface and you want to run some commands there. I don’t think that’s possible. Can you try running the command without pulling up the interface? eg bwcli command (the command here would be the one you are planning to run after you open the interface). You can run bwcli –help to show up the usage options.

        Alternatively, you can also try pexpect module for spawning child applications.

        Reply

    5. This is useless for python 3. You are stuck in the past of python 2. You should modernize to python 3. It’s getting very frustrating to use python at all when so many programmers are still stuck in the past!

      Reply

        1. In Python 3 print() is a function and the syntax “except Exception,e”
          It’s 2020; your example (from a QA company would be much better if it were up to date!

    6. Hi Indira,

      Love the example and explanation!

      Is there a way I can pass multiple commands to the SSH instead of just one command through the COMMAND list in config file? Currently I can do this via adding a semicolon between commands. But I was wondering if there is a way I can have multiple instances of this COMMAND list?

      Thanks!

      Nik

      Reply

      1. Nik,

        The command list is an array. Each element is either a single command or a semi-colon separated group of commands. You can see one example of the conf associated with this example here: https://github.com/qxf2/qxf2-page-object-model/blob/master/conf/ssh_conf.py

        You can have multiple instances of the command list. In fact, we ourselves don’t read commands from the conf file as shown in this simplified examples. Instead, the way we usually use this script is to simply call

        ssh_obj.execute_command(['my_command1'])
result = ssh_obj.ssh_output
#Some logic based on result
ssh_obj.execute_command(['my_command2'])

        Hope this answered your question.

        Reply

    7. Hi,
      I have Bluecoat packet shaper device, which will ask username 2 times then password.. how do I connect this device…can any help me please??

      Reply

      1. Hi
        Does it ask for username again after authenticating using username & password? . Did you try authenticating using SSH key pair?

        Reply

    8. When I run this python script my command prompt simply returns blank, i’m also beginner in python, what could be the cause?

      Reply

      1. Bharath,
        Can you provide more details on how you are running the script? Did you specify the required credentials and .pem file details in ssh_conf.py file?

        Reply

        1. Thanks for replying!
          I figured it out, I wasn’t knowing till then that I had to call the main function (The ___name__ because I had not copied the last part of the script, soon as it was added the script executed function and everything in it.

    9. Hi, Maybe I’m missing something due to lack of knowledge, but something is not working:
      I’m successfully connecting to my server with its IP address, user and password:
      Establishing an SSH connection
      Connected to the server 10.X.X.X

      But then running the commands doesn’t work:
      Executing command –> show version
      Command timed out. show version
      Unable to execute the commands

      This is a legit and working command which works when I’m using external SSH clients (such as Putty, Secure CRT).

      Any idea why that wouldn’t work?
      Thanks

      Reply

      1. Raz,
        Looks like ‘show version’ is not a built-in shell command and hence the command timed out. Did you try executing any other commands?

        Reply

        1. Hi, thanks for responding.
          Show version is a built-in command in the device I’m trying to perform the command (NX9510) from Motorola (Extreme) production.
          The command is working when I’m using it in Secure CRT.

    10. Hi,
      Thank you Indira for this post.
      I am trying to use python for test automation of our Unix application. Our application has command line interface and it relies on a few function keys for navigation (e.g. F11 to select menu, F10 for going back, UP / DOWN arrow for navigation in menu). I’m able to send the Unix commands through Python / Paramiko and launch the application in Unix. But, how can I send the special keys (e.g. F9, F10, Up/Down arrow) to Unix using Python ? can you pls. help or provide pointers on this. Thanks..

      Reply

    Leave a Reply

    Your email address will not be published. Required fields are marked *