SSH using Python Paramiko

Recently, as part of an automated test, we needed to SSH into a server, toggle a service and then check the response on a web application. We used the Python module Paramiko. We ended up writing simple wrappers around the most common actions test automation may need to perform over SSH. In this post, we will share the same.

We have written this tutorial for absolute beginners. If you are already familiar with SSH (there are people who are not!) and comfortable with Python, please jump ahead to the section ‘Putting it all together’.

Note 1: We have also integrated this code into our open-sourced Python test automation framework.
Note 2: The Python 3 code for this article is here

How SSH Works

The SSH connection is implemented using a client-server model. To establish a connection the server should be running and clients generally authenticated either using passwords or SSH keys. Password Authentication is simple and straightforward. To authenticate using SSH keys, a user must have an SSH key pair (Public and Private key). On the remote server, the list of public keys is maintained (usually) in the ~/.ssh/authorized_keys directory. When the client connects to the remote server using the public key, the server checks for it and sends an encrypted message which can only be decrypted with the associated private key at the client side. We will be using a Python module called Paramiko. The Paramiko module gives an abstraction of the SSHv2 protocol with both the client side and server side functionality. As a client, you can authenticate yourself using a password or key and as a server, you can decide which users are allowed access and the channels you allow.

Installing Paramiko

Installing Paramiko is straightforward, it has only one direct hard dependency: the Cryptography library. If pip is above 8.x, nothing else is required. pip will install statically compiled binary archives of Cryptography its dependencies. For more details on installation, please refer to this Paramiko’s documentation.

To install Paramiko on Windows

pip install paramiko

Overview of the Parmiko tutorial

In the coming sections, we will be talking about below items:

  • Connecting to the remote server using Paramiko
  • Executing commands on the server
  • File transfers
  • Putting it all together

    • We have written different methods for each functionality. All the parameters are read from a configuration file. We are reading the parameters like credentials, commands & file paths from the conf file because when there is any change in the machine or any credential changes, changing the configuration file alone is enough instead of changing the script.

    After importing Paramiko, to make everything more modular, we created a class called Ssh_Util and wrote different methods for each functionality.

    Connecting to the remote server using Paramiko

    Firstly, we created an initialization function inside the class and initialized required variables. Some of the parameters like host, username, password etc., are read from a configuration file.

    def __init__(self):
        self.ssh_output = None
        self.ssh_error = None
        self.client = None
        self.host= conf_file.HOST
        self.username = conf_file.USERNAME
        self.password = conf_file.PASSWORD
        self.timeout = float(conf_file.TIMEOUT)
        self.commands = conf_file.COMMANDS
        self.pkey = conf_file.PKEY
        self.port = conf_file.PORT
        self.uploadremotefilepath = conf_file.UPLOADREMOTEFILEPATH
        self.uploadlocalfilepath = conf_file.UPLOADLOCALFILEPATH
        self.downloadremotefilepath = conf_file.DOWNLOADREMOTEFILEPATH
        self.downloadlocalfilepath = conf_file.DOWNLOADLOCALFILEPATH

    We created a function connect() and placed inside the Ssh_Util class. This function will be used to connect to the remote server.

    def connect(self):
        "Login to the remote server"
        try:
            #Paramiko.SSHClient can be used to make connections to the remote server and transfer files
            print("Establishing ssh connection")
            self.client = paramiko.SSHClient()
            #Parsing an instance of the AutoAddPolicy to set_missing_host_key_policy() changes it to allow any host.
            self.client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
            #Connect to the server
            if (self.password == ''):
                private_key = paramiko.RSAKey.from_private_key_file(self.pkey)
                self.client.connect(hostname=self.host, port=self.port, username=self.username, pkey=private_key, timeout=self.timeout, allow_agent=False,look_for_keys=False)
                print("Connected to the server",self.host)
            else:
                self.client.connect(hostname=self.host, port=self.port, username=self.username, password=self.password,timeout=self.timeout, allow_agent=False,look_for_keys=False)    
                print("Connected to the server",self.host)
        except paramiko.AuthenticationException:
            print("Authentication failed, please verify your credentials")
            result_flag = False
        except paramiko.SSHException as sshException:
            print("Could not establish SSH connection: %s" % sshException)
            result_flag = False
        except socket.timeout as e:
            print("Connection timed out")
            result_flag = False
        except Exception,e:
            print("Exception in connecting to the server")
            print("PYTHON SAYS:",e)
            result_flag = False
            self.client.close()
        else:
            result_flag = True
 
        return result_flag

    Firstly, we created an instance ‘client’ of paramiko.SSHClient(). Paramiko.SSHClient is the primary client used to make connections to the remote server and execute commands. This class wraps Transport, Channel, and SFTPClient to take care of most aspects of authenticating and opening channels. The creation of an SSHClient object allows establishing server connections via the connect() method.

    When we try to connect to a server for the first time, we see an error message prompted saying that the machine is not informed about the remote server that we are trying to access. Paramiko’s client.set_missing_host_key_policy is the method used for this purpose. By default, the paramiko.SSHclient sets the policy to the Reject policy which means the policy rejects connection without validating. In our code, we are overriding this by passing AutoAddPolicy() function where the new server will automatically add the server’s host key without prompting. Note:- In a testing environment, we can use set_missing_host_key_policy and set AutoAddPolicy but for security purpose, this is not a good idea to use in production.

    Paramiko supports both password-based authentication and key-pair based authentication for accessing the server. In our code, we are checking for a password and if available, authentication is attempted using plain username/password authentication. If the password is not available, authentication is attempted by reading the private key file.

    client.Connect() is to connect to an SSH server. This method also allows to provide our own private key, or connect to the SSH agent on the local machine or read from the user’s local key files. We also used timeout (in seconds) to wait for an authentication response. We are capturing a couple of exceptions like paramiko.AuthenticationException, paramiko.SSHException, Socket.timeout to handle the errors while connecting.

    Executing commands on the server

    Now, you are connected to the remote server. The next step is to execute commands on the SSH server. To run a command on the server the exec_command() function is called on the SSHClient with the command passed as input. When you execute commands using exec_command a new Channel is opened and the requested command is executed. The response is returned as Python file-like objects representing stdin, stdout, and stderr(as a 3-tuple)

    • The stdin is a write-only file which can be used for input commands.
    • The stdout file give the output of the command.
    • The stderr gives the errors returned on executing the command. Will be empty if there is no error.

    We have written below function execute_command() to execute the commands. The input for this function is the set of commands. The call to the function connect() is made first and once the connection is established it is followed by a call to function exec_command(). We are storing the stdout and stderr into two variables namely ssh_output and ssh_error respectively. The ssh_output is the output to standard output as produced when the command executed on the remote computer. The ssh_error is the output of standard error produced at the same occasion. The user can use these variables for further processing if needed.

    def execute_command(self,commands):
        """Execute a command on the remote host.Return a tuple containing
        an integer status and a two strings, the first containing stdout
        and the second containing stderr from the command."""
        self.ssh_output = None
        result_flag = True
        try:
            if self.connect():
                for command in commands:
                    print("Executing command --> {}".format(command))
                    stdin, stdout, stderr = self.client.exec_command(command,timeout=10)
                    self.ssh_output = stdout.read()
                    self.ssh_error = stderr.read()
                    if self.ssh_error:
                        print("Problem occurred while running command:"+ command + " The error is " + self.ssh_error)
                        result_flag = False
                    else:    
                        print("Command execution completed successfully",command)
                    self.client.close()
            else:
                print("Could not establish SSH connection")
                result_flag = False   
        except socket.timeout as e:
            print("Command timed out.", command)
            self.client.close()
            result_flag = False                
        except paramiko.SSHException:
            print("Failed to execute the command!",command)
            self.client.close()
            result_flag = False    
 
        return result_flag

    File Transfers

    File transfers are needed to perform remote file operations. Paramiko allows to programmatically send and receive files using the SFTP protocol, the connection with the remote host is established in the same way explained in the previous sections, the call to connect() is followed by a call to open_sftp() that returns a new SFTPClient session object. This object allows to perform common SFTP operations like get(), put(),listdir().

    We have created two functions(upload_file() and download_file()) to upload a file to the remote server and download a file from the remote server.

    Firstly, we created a function upload_file() which uploads a file to the remote server. The code below establishes the SFTP Connection using the SSH client and uploads a file. The put() method will copy a local file (local path) to the SFTP server as the remote path. Note:- The filename should be included. Only specifying a directory may result in an error. Once the operation is done you may close the SFTP session and its underlying channel using ftp_client.close().

    def upload_file(self,uploadlocalfilepath,uploadremotefilepath):
        "This method uploads the file to remote server"
        result_flag = True
        try:
            if self.connect():
                ftp_client= self.client.open_sftp()
                ftp_client.put(uploadlocalfilepath,uploadremotefilepath)
                ftp_client.close() 
                self.client.close()
            else:
                print("Could not establish SSH connection")
                result_flag = False  
        except Exception,e:
            print('\nUnable to upload the file to the remote server',uploadremotefilepath)
            print('PYTHON SAYS:',e)
            result_flag = False
            ftp_client.close()
            self.client.close()
 
        return result_flag

    Similarly, download_file() function downloads a file from the remote server. The code below establishes the SFTP Connection using the SSH client and downloads a file. The get() method will copy a remote file (remote path) from the SFTP server to the local host as local path. Once the operation is done you may close the SFTP session and its underlying channel using ftp_client.close().

    def download_file(self,downloadremotefilepath,downloadlocalfilepath):
        "This method downloads the file from remote server"
        result_flag = True
        try:
            if self.connect():
                ftp_client= self.client.open_sftp()
                ftp_client.get(downloadremotefilepath,downloadlocalfilepath)
                ftp_client.close()  
                self.client.close()
            else:
                print("Could not establish SSH connection")
                result_flag = False  
        except Exception,e:
            print('\nUnable to download the file from the remote server',downloadremotefilepath)
            print('PYTHON SAYS:',e)
            result_flag = False
            ftp_client.close()
            self.client.close()
 
        return result_flag

    Putting it all together

    Note: The Python 3 code for this article is here. Please use it instead of the file below.

    import paramiko
import os,sys,time
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from conf import ssh_conf as conf_file   
import socket
 
class Ssh_Util:
    "Class to connect to remote server" 
 
    def __init__(self):
        self.ssh_output = None
        self.ssh_error = None
        self.client = None
        self.host= conf_file.HOST
        self.username = conf_file.USERNAME
        self.password = conf_file.PASSWORD
        self.timeout = float(conf_file.TIMEOUT)
        self.commands = conf_file.COMMANDS
        self.pkey = conf_file.PKEY
        self.port = conf_file.PORT
        self.uploadremotefilepath = conf_file.UPLOADREMOTEFILEPATH
        self.uploadlocalfilepath = conf_file.UPLOADLOCALFILEPATH
        self.downloadremotefilepath = conf_file.DOWNLOADREMOTEFILEPATH
        self.downloadlocalfilepath = conf_file.DOWNLOADLOCALFILEPATH
 
    def connect(self):
        "Login to the remote server"
        try:
            #Paramiko.SSHClient can be used to make connections to the remote server and transfer files
            print("Establishing ssh connection")
            self.client = paramiko.SSHClient()
            #Parsing an instance of the AutoAddPolicy to set_missing_host_key_policy() changes it to allow any host.
            self.client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
            #Connect to the server
            if (self.password == ''):
                self.pkey = paramiko.RSAKey.from_private_key_file(self.pkey)
                self.client.connect(hostname=self.host, port=self.port, username=self.username,pkey=self.pkey ,timeout=self.timeout, allow_agent=False, look_for_keys=False)
                print("Connected to the server",self.host)
            else:
                self.client.connect(hostname=self.host, port=self.port,username=self.username,password=self.password,timeout=self.timeout, allow_agent=False, look_for_keys=False)    
                print("Connected to the server",self.host)
        except paramiko.AuthenticationException:
            print("Authentication failed, please verify your credentials")
            result_flag = False
        except paramiko.SSHException as sshException:
            print("Could not establish SSH connection: %s" % sshException)
            result_flag = False
        except socket.timeout as e:
            print("Connection timed out")
            result_flag = False
        except Exception,e:
            print('\nException in connecting to the server')
            print('PYTHON SAYS:',e)
            result_flag = False
            self.client.close()
        else:
            result_flag = True
 
        return result_flag    
 
    def execute_command(self,commands):
        """Execute a command on the remote host.Return a tuple containing
        an integer status and a two strings, the first containing stdout
        and the second containing stderr from the command."""
        self.ssh_output = None
        result_flag = True
        try:
            if self.connect():
                for command in commands:
                    print("Executing command --> {}".format(command))
                    stdin, stdout, stderr = self.client.exec_command(command,timeout=10)
                    self.ssh_output = stdout.read()
                    self.ssh_error = stderr.read()
                    if self.ssh_error:
                        print("Problem occurred while running command:"+ command + " The error is " + self.ssh_error)
                        result_flag = False
                    else:    
                        print("Command execution completed successfully",command)
                    self.client.close()
            else:
                print("Could not establish SSH connection")
                result_flag = False   
        except socket.timeout as e:
            print("Command timed out.", command)
            self.client.close()
            result_flag = False                
        except paramiko.SSHException:
            print("Failed to execute the command!",command)
            self.client.close()
            result_flag = False    
 
        return result_flag
 
    def upload_file(self,uploadlocalfilepath,uploadremotefilepath):
        "This method uploads the file to remote server"
        result_flag = True
        try:
            if self.connect():
                ftp_client= self.client.open_sftp()
                ftp_client.put(uploadlocalfilepath,uploadremotefilepath)
                ftp_client.close() 
                self.client.close()
            else:
                print("Could not establish SSH connection")
                result_flag = False  
        except Exception,e:
            print('\nUnable to upload the file to the remote server',uploadremotefilepath)
            print('PYTHON SAYS:',e)
            result_flag = False
            ftp_client.close()
            self.client.close()
 
        return result_flag
 
    def download_file(self,downloadremotefilepath,downloadlocalfilepath):
        "This method downloads the file from remote server"
        result_flag = True
        try:
            if self.connect():
                ftp_client= self.client.open_sftp()
                ftp_client.get(downloadremotefilepath,downloadlocalfilepath)
                ftp_client.close()  
                self.client.close()
            else:
                print("Could not establish SSH connection")
                result_flag = False  
        except Exception,e:
            print('\nUnable to download the file from the remote server',downloadremotefilepath)
            print('PYTHON SAYS:',e)
            result_flag = False
            ftp_client.close()
            self.client.close()
 
        return result_flag
 
 
#---USAGE EXAMPLES
if __name__=='__main__':
    print("Start of %s"%__file__)
 
    #Initialize the ssh object
    ssh_obj = Ssh_Util()
 
    #Sample code to execute commands
    if ssh_obj.execute_command(ssh_obj.commands) is True:
        print("Commands executed successfully\n")
    else:
        print("Unable to execute the commands")
 
    """
    #Sample code to upload a file to the server
    if ssh_obj.upload_file(ssh_obj.uploadlocalfilepath,ssh_obj.uploadremotefilepath) is True:
        print("File uploaded successfully", ssh_obj.uploadremotefilepath)
    else:
        print("Failed to upload the file")
 
    #Sample code to download a file from the server
    if ssh_obj.download_file(ssh_obj.downloadremotefilepath,ssh_obj.downloadlocalfilepath) is True:
        print("File downloaded successfully", ssh_obj.downloadlocalfilepath)
    else:
        print("Failed to download the file")
    """

    Hope this code helps you get started with Paramiko easily. Happy coding! If you liked this article and want to learn more about Qxf2 and our testing services for startups, click here.

    References

    1) SSH and transfer file using paramiko
    2) How to SSH in Python using paramiko
    3) Paramiko Github examples
    4) How does SSH works?

    Related posts:

    Default ThumbnailGetting started with MongoDB and Python Default ThumbnailGet started with SQLite and Python Default ThumbnailReporting automation results to TestRail using Python
    Indira Nellutla

    64 thoughts on “SSH using Python Paramiko”

    1. Hi,

      Do I need to add OpenSSH feature in windows to run this module? I am trying to communicate between two Windows 10 PC connected with LAN.

      First I got: socket.error: [Errno 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
      -> This seemed to be because of firewall.

      Disabling firewall, I get this error:
      raise NoValidConnectionError(error)
      paramiko.ssh_exception.NoValidConnectionError: [Errno None] Unable to connect to port 22 on {IP ADDRESS OF HOST THAT I AM TRYING TO CONNECT}

      Reply

      1. Hi Juhana,
        I am not so sure how to resolve the issue which you are facing. We don’t need OpenSSH client to run the module. Guess you can try to isolate connection error first by trying to ssh using bash or putty or OpenSSH.

        Reply

        1. Hi,

          I figured out the problem. There was no ssh server listening in the remote pc. I had to install openssh server and allow port 22 for communication in Windows Firewall.

    2. Hi,

      My issue is funny character that is returned in stdout.readlines() from exec_command.

      This is what i found in other forum:
      stdout._set_mode(‘b’)
      opt = stdout.readlines()
      opt = “”.join(opt)
      print opt
      Still, I get some funny characters, example when exec_command(‘dir’).

      without: _set_mode(‘b’) readlines() gives me this error:
      UnicodeDecodeError: ‘utf8’ codec can’t decode byte 0xff

      Reply

    3. Hi,
      I am new to python coding. I came across this issue and am stuck there. I am connecting to a server using my user_id and password. Post this i am supposed to do a “be functional_id” to get into the functional Id amd then execute commands. But i am not able to do this. It is staying with the user it logged in with. How do i execute “be functional_id” from my code and get into the functional id. Hope i was clear with the issue.

      Reply

      1. Hi,
        Are you trying to switch between user & a service account on Linux?
        Did you try using sudo su -s /bin/bash command?

        Reply

    5. Hi,

      I need to install application in remote pc. I am using paramiko sftp to transfer images. Then I invoke extraction in remote pc with exec_command(). This works fine. But the application for some reason does not install.

      I am passing path of installer with arguments required to do installation silently, in exec_command(). This does not work. Then I tried different approach by creating installer script separately, uploading it to remote machine and calling ‘python {scriptfilename.py}’ . This does not work as well. I have tried running script file separately, it works but invoking script file in exec_command() seems to have some problem.

      What might be the cause? Although I thought the installation should start from exec_command() without additional script file to make the installation happen, as in exec command I am just passing installer location and argument, Something is preventing it from happening.

      In separate script file, I have implemented python subprocess popen to open the installer. Again, it runs as desired when I am executing only this file. Running this script file from paramiko exec_command() does nothing.

      What am I missing?

      Reply

      1. Hi Shiv,
        I guess the approach you followed should work. Is it possible for you to share the script file so that i can debug it better?

        Reply

    6. Hi All,
      Could anyone suggest me for below ?:
      i am running a script located on remote linux server from Jumphost using paramiko.
      however it is hanging while script is asking for user input. in script I am just asking input via raw_input.

      any suggestion please ??

      Thanks in advance.

      Reply

      1. Deepak,

        We haven’t tried this but you can try using -u option in the command where you are executing python script. For eg:

        stdout = client.exec_command('python -u script.py')

        Some notes found regarding raw_input:
        If you use raw_input in ssh, it does not flush stdout before reading from stdin. It saves the output until the script is finished and only then prints everything in the buffer. Since stdout will not be flushed the user will not know he is being prompted. So you need to run python unbuffered(-u) which will force stdout to flush after every output.

        Ref:- https://stackoverflow.com/questions/35227230/python-script-not-able-to-read-user-input-when-run-remotely

        Hope this helps!

        Reply

    7. sshclient exec_command is taking long time ? what might be the reason and it will very helpful if u can tell me how how to speed it up

      Reply

    8. sshclient exec_command is taking long time,what might be the reason and it would be very helpful if suggest how to speed it up

      Reply

    9. How any one can execute a sudo command using paramiko??It is giving an error ”sudo: no tty present and no askpass program specified”

      Reply

      1. Hi,
        I guess adding the -t option to your ssh command may help. You can refer to this link for more information.

        Reply

    10. Hi, Maybe I’m missing something due to lack of knowledge in python, but something is not working:
      I’m successfully connecting to my server for (equipment access service “EAS”) with its IP address, user and password:
      Establishing an SSH connection
      Connected to the server xx.xx.xx.xxx

      I do not know what (# .pem file details PKEY = ‘Enter your key filename here’) in the ssh-conf.py file

      But then running the commands doesn’t work:

      COMMANDS = [‘ena; show version’]

      Executing command –> ena
      ( to access the enable mode)
      Executing command –> show version
      the Commands timed out. show version
      Unable to execute the commands (‘Problem occurred while running command:ena The error is /bin/bash: line 1: ena: command not found’)

      This is a legit and working command which works when I’m using external SSH clients on (such as Putty,).

      Any idea why that wouldn’t work?
      Thanks

      Reply

      1. Hi,
        You can also connect to the server using .pem file if you do not have a password-based authentication. If the password is not available, authentication is attempted by reading the private key file which you define in the ssh_conf.py. Regarding commands not getting executed, I am not too sure if ‘ena’ and ‘show version’ are built-in shell commands. Are you able to execute any other commands?

        Thanks
        Indira Nellutla

        Reply

        1. Hi, thanks for responding.

          I do not have a .pem file
          the problem that it is for return any import commands must be executed the command (> ena)

          the function “channel = ssh.invoke_shell ()” it works well

          I have a little code that gives me the results

          “channel = ssh.invoke_shell ()
          channel.send ( “ena \ n”)
          time.sleep (2)
          channel.send ( “\ n”)
          time.sleep (2)
          channel.send (“show version \ n”)
          time.sleep (2)
          while not channel.recv_ready ():
          print “Working …”
          time.sleep (2)
          print channel.recv (1024)
          channel.send (“show version”)
          while not channel.recv_ready ():
          print “Authenticating ….”
          time.sleep (2)
          buff = channel.recv (1024)
          print buff
          time.sleep (2) ”

          but I would like to use the function exec_command (‘xxx’) to modify all the code.

          Think you.

    Leave a Reply

    Your email address will not be published. Required fields are marked *