Getting started with MongoDB and Python

I was recently playing around with MongoDB. It took me a little while to get setup and write some simple Python scripts to access it. I thought I would write up a summary and share my learning.

Why this post?

Companies have begun adopting MongoDB at a good clip. I think, interacting with a NoSQL database will soon become a regular action for most testers. I am summarizing my learning so testers who are new to MongoDB can save time by having one place to get setup and understand how to interact with MongoDB.

What is MongoDB?

MongoDB is a NoSQL database. MongoDB does not have the concept of tables static columns. Instead MongoDB is a set of ‘collections’. Each ‘collection’ is made of ‘documents’. The documents are very similar to JSON. MongoDB is useful when you don’t know the exact structure of your data upfront. It is also useful when you don’t want to specify your schema upfront. MongoDB makes it extremely easy to modify new fields (have you tried adding new columns in RDBMS land?). While I did do a lot more reading online to get comfortable with the idea, at this stage of my learning, the key takeaways for me were:

  • the database is organized into documents and collections
  • no strict enforcement of table constraints

Tutorial overview

In this tutorial, we’ll cover installation, setup and basic CRUD operations. You need access to an Ubuntu machine to follow along.

1. Install MongoDB on Ubuntu
To install Mongo on Ubuntu, you need to first let the Aptitude package manager know where to look for the MongoDB installer. You can do that by running these commands:

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10
echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' | sudo tee /etc/apt/sources.list.d/mongodb.list
sudo apt-get update

You can then install MongoDB with:

sudo apt-get install -y mongodb-org

By default, MongoDB binds itself to 127.0.0.1 and port 27017. It stores its data files in /var/lib/mongodb and its configuration file is in /etc/mongod.config.

You can test your installation by simply typing in mongo (or ‘sudo mongo’) in your terminal. You should see a something like the image below if things went well.

mongodb install

Reference: Liquid Web: How to install MongoDB

2. Start and stop MongoDB
You can start|stop|restart MongoDB like this:

sudo service mongod start
sudo service mongod stop
sudo service mongod restart

3. Create a new database
Mongo uses the use command to create a database if it is not already present. Sigh – I found this pretty confusing. Pull up your MongoDB shell by typing mongo in your terminal. Let us create a database called qxf2:

use qxf2

If things went well, you will see the message ‘switched to db qxf2’

4. Create a new collection
A collection is something similar to a table in SQL. Well, not exactly – but it is a useful mental model to have when you start. Let us create a collection called Employees by typing the command below in your Mongo shell:

db.createCollection('Employees')

If things went well, you will see the message ‘{ “ok” : 1 }’

5. Add a new user
Its easy to add a user to MongoDB through the Mongo shell. Let us add me as a user to the qxf2 database.You simply need something similar to the following command:

db.createUser({"user":"raji", "pwd":"Testdb123", "roles":[{"role":"dbAdmin","db":"qxf2"}]})

If things went well, you will see the message ‘Successfully added user:’

Now you are setup to learn how to connect to your MongoDB database using Python. But I strongly encourage you to revisit the Mongo shell and try other commands too!

6. Install PyMongo
PyMongo is the recommended Python module to work with MongoDB. You can install it by:

pip install –U pymongo

7. Connect to MongoDB
There are a couple of ways to connect to the MongoDB instance using Python. We’ll show you both.

import pymongo
 
host = "127.0.0.1"
port = "27017"
user = "raji"
password = "Testdb123"
db_name = "qxf2"
auth_mechanism = "MONGODB-CR"
uri = "mongodb://"+user+":"+password+"@"+host+":"+port+"/"+db_name+"?"+"authMechanism="+auth_mechanism
client = pymongo.MongoClient(uri) #client object to connect
 
#Logic to obtain the db object
db = None
if db_name in client.database_names():
    db = client[db_name] 
#And now you have a db object to interact with

The other way to go about connecting is:

import pymongo
 
host = "127.0.0.1"
port = "27017"
user = "raji"
password = "Testdb123"
db_name = "qxf2"
 
uri = host+":"+port
client = pymongo.MongoClient(uri)
db = client[db_name]
db.authenticate(user,password)
#And now you have a db object to interact with

8. Insert a new document
We have created a collection called ‘Employees’ already. Let us insert a document into a collection with

db = client[database] 
collection = db[collection_name] 
collection.insert(document)

9. Update an existing document
You can update an existing document like this:

db = client[database] 
collection = db[collection_name] 
collection.update(doc_key,doc_update)

10. Delete a document
You probably guessed the call to delete a document looks like this:

db = client[database] 
collection = db[collection_name] 
collection.remove(doc_key)

11. Putting it all together
Let’s put it all together into a useful MongoDriver class.

"""
Qxf2 Services: Utility script to connect to MongoDB
NOTE: This was written up for a blog post/tutorial 
We use a cleaned up and more robust version at clients
"""
 
import pymongo
 
class Mongo_Driver:
    "Class to connect to MongoDB"
    def __init__(self,host,port):
        "Constructor"
        self.host = host
        self.port = port
 
 
    def connect(self,database,username,password,auth_mechanism=None):
        "Connect to MongoDB database"
        #src: https://docs.mongodb.org/manual/core/authentication/#security-authentication-mechanisms
        if auth_mechanism is None:
            self.auth_mechanism = "MONGODB-CR"#by default authMechanism=MONGODB-CR , change if required
        else:
            self.auth_mechanism = auth_mechanism
 
        #Construct the connection URI "mongodb://user:password@hostnameOrIP:port/database?authMechanism=MONGODB-CR"
        self.uri = "mongodb://"+username+":"+password+"@"+self.host+":"+self.port+"/"+database+"?"+"authMechanism="+self.auth_mechanism
        self.client = pymongo.MongoClient(self.uri)
        self.db = None
        result_flag = False
        try:
            if database in self.client.database_names():
                self.db = self.client[database] 
            if self.db is not None:
                result_flag = True
        except Exception,e:
            print '\nException when connecting to Mongo instance'
            print 'PYTHON SAYS:'
            print e
            print '\n'
 
        return result_flag
 
 
    def connect2(self,database,username,password):
        "Another way to connect to the database"
        self.uri = self.host+":"+self.port
        self.client = pymongo.MongoClient(self.uri)
        self.db = self.client[database]
        result_flag = self.db.authenticate(username,password)
 
        return result_flag
 
 
    def insert_document(self,document,collection_name):
        "Insert a document into a collection"
        result_flag = False
        try:
            self.collection = self.get_collection(collection_name)
            self.collection.insert(document)
        except Exception,e:
            print '\nException in insert_document'
            print 'PYTHON SAYS:'
            print e
            print '\n'
        else:
            result_flag = True
 
        return result_flag
 
 
    def get_collection(self,collection_name):
        "Return the collection object with name collection_name"
        self.collection = None
        try:
            self.collection = self.db[collection_name] 
        except Exception,e:
            print "Collection %s not found"%collection_name
 
        return self.collection
 
 
    def update_document(self,doc_key,doc_update,collection_name):   
        "Update a document based on its key"
        result_flag = False
        try:
            self.collection = self.get_collection(collection_name)
            result_obj = self.collection.update(doc_key,doc_update)
            if result_obj['updatedExisting'] is True:
                result_flag = True
        except Exception,e:
            print '\nException in update_document'
            print 'PYTHON SAYS:'
            print e
            print '\n'
 
        return result_flag
 
 
    def delete_document(self,doc_key,collection_name):
        "Delete a given document"
        result_flag = False
        try:
            self.collection = self.get_collection(collection_name)            
            result_obj = self.collection.remove(doc_key)
            if result_obj['n'] is 1:
                result_flag = True
        except Exception,e:
            print '\nException in delete_document'
            print 'PYTHON SAYS:'
            print e
            print '\n'
        else:
            return result_flag
 
 
#---EXAMPLE USAGE---
if __name__=='__main__':    
    'user created with role:readWrite - Example: db.createUser({"user":"raji1","pwd":"Testdb123","roles":[{"role":"readWrite" OR "dbAdmin","db":"qxf2"}]}) '
    #mongo connection string parameters (read these from config file)
    host = "127.0.0.1"
    port = "27017"
    user = "raji"
    password = "Testdb123"
    db_name = "qxf2"
    collection = "Employees"
    document = {"name":"Jack","address":[{"area":"NRI Layout","city":"blr","pin":"560016","state":"KA"},{"area":"NRI Layout","city":"blr","pin":"560016","state":"KA"}],"contact":"90000000006","emp_id":"0007"}
    document_key = {"name":"Jack"}
 
    #initialize mongo driver object
    mongo_obj = Mongo_Driver(host,port)
 
    #connect to a db
    if mongo_obj.connect(db_name,user,password) is True:
        print "DB authorization successful"
        #insert document
        if mongo_obj.insert_document(document,collection) is True:
            print "insert document successful"
        else:
            print "insert document unsuccessful"
 
        #update document
        if mongo_obj.update_document(document_key,document,collection) is True:
            print "update document successful"
        else:
           print "update document unsuccessful"
 
        #delete document
        if mongo_obj.delete_document(document_key,collection) is True:
            print "delete document successful"
        else:
            print "delete document unsuccessful"
    else:
        print "DB authorization unsuccessful"

I hope this helps you get started with MongoDB. Questions? Comments? Post them in the comments section below and I will get back to you soon.

Related posts:

Default ThumbnailSSH using Python Paramiko Default ThumbnailReporting automation results to TestRail using Python Default ThumbnailGet started with SQLite and Python
Rajeswari Gali

One thought on “%1$s”

  1. Pingback: Testing Bits – 10/25/15 – 10/31/15 | Testing Curator Blog

Leave a Reply

Your email address will not be published. Required fields are marked *