I was recently playing around with MongoDB. It took me a little while to get setup and write some simple Python scripts to access it. I thought I would write up a summary and share my learning.
Why this post?
Companies have begun adopting MongoDB at a good clip. I think, interacting with a NoSQL database will soon become a regular action for most testers. I am summarizing my learning so testers who are new to MongoDB can save time by having one place to get setup and understand how to interact with MongoDB.
What is MongoDB?
MongoDB is a NoSQL database. MongoDB does not have the concept of tables static columns. Instead MongoDB is a set of ‘collections’. Each ‘collection’ is made of ‘documents’. The documents are very similar to JSON. MongoDB is useful when you don’t know the exact structure of your data upfront. It is also useful when you don’t want to specify your schema upfront. MongoDB makes it extremely easy to modify new fields (have you tried adding new columns in RDBMS land?). While I did do a lot more reading online to get comfortable with the idea, at this stage of my learning, the key takeaways for me were:
- the database is organized into documents and collections
- no strict enforcement of table constraints
Tutorial overview
In this tutorial, we’ll cover installation, setup and basic CRUD operations. You need access to an Ubuntu machine to follow along.
1. Install MongoDB on Ubuntu
To install Mongo on Ubuntu, you need to first let the Aptitude package manager know where to look for the MongoDB installer. You can do that by running these commands:
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10 echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' | sudo tee /etc/apt/sources.list.d/mongodb.list sudo apt-get update |
You can then install MongoDB with:
sudo apt-get install -y mongodb-org |
By default, MongoDB binds itself to 127.0.0.1 and port 27017. It stores its data files in /var/lib/mongodb and its configuration file is in /etc/mongod.config.
You can test your installation by simply typing in mongo (or ‘sudo mongo’) in your terminal. You should see a something like the image below if things went well.
Reference: Liquid Web: How to install MongoDB
2. Start and stop MongoDB
You can start|stop|restart MongoDB like this:
sudo service mongod start sudo service mongod stop sudo service mongod restart |
3. Create a new database
Mongo uses the use command to create a database if it is not already present. Sigh – I found this pretty confusing. Pull up your MongoDB shell by typing mongo in your terminal. Let us create a database called qxf2:
use qxf2 |
If things went well, you will see the message ‘switched to db qxf2’
4. Create a new collection
A collection is something similar to a table in SQL. Well, not exactly – but it is a useful mental model to have when you start. Let us create a collection called Employees by typing the command below in your Mongo shell:
db.createCollection('Employees') |
If things went well, you will see the message ‘{ “ok” : 1 }’
5. Add a new user
Its easy to add a user to MongoDB through the Mongo shell. Let us add me as a user to the qxf2 database.You simply need something similar to the following command:
db.createUser({"user":"raji", "pwd":"Testdb123", "roles":[{"role":"dbAdmin","db":"qxf2"}]}) |
If things went well, you will see the message ‘Successfully added user:’
Now you are setup to learn how to connect to your MongoDB database using Python. But I strongly encourage you to revisit the Mongo shell and try other commands too!
6. Install PyMongo
PyMongo is the recommended Python module to work with MongoDB. You can install it by:
pip install –U pymongo |
7. Connect to MongoDB
There are a couple of ways to connect to the MongoDB instance using Python. We’ll show you both.
import pymongo host = "127.0.0.1" port = "27017" user = "raji" password = "Testdb123" db_name = "qxf2" auth_mechanism = "MONGODB-CR" uri = "mongodb://"+user+":"+password+"@"+host+":"+port+"/"+db_name+"?"+"authMechanism="+auth_mechanism client = pymongo.MongoClient(uri) #client object to connect #Logic to obtain the db object db = None if db_name in client.database_names(): db = client[db_name] #And now you have a db object to interact with |
The other way to go about connecting is:
import pymongo host = "127.0.0.1" port = "27017" user = "raji" password = "Testdb123" db_name = "qxf2" uri = host+":"+port client = pymongo.MongoClient(uri) db = client[db_name] db.authenticate(user,password) #And now you have a db object to interact with |
8. Insert a new document
We have created a collection called ‘Employees’ already. Let us insert a document into a collection with
db = client[database] collection = db[collection_name] collection.insert(document) |
9. Update an existing document
You can update an existing document like this:
db = client[database] collection = db[collection_name] collection.update(doc_key,doc_update) |
10. Delete a document
You probably guessed the call to delete a document looks like this:
db = client[database] collection = db[collection_name] collection.remove(doc_key) |
11. Putting it all together
Let’s put it all together into a useful MongoDriver class.
""" Qxf2 Services: Utility script to connect to MongoDB NOTE: This was written up for a blog post/tutorial We use a cleaned up and more robust version at clients """ import pymongo class Mongo_Driver: "Class to connect to MongoDB" def __init__(self,host,port): "Constructor" self.host = host self.port = port def connect(self,database,username,password,auth_mechanism=None): "Connect to MongoDB database" #src: https://docs.mongodb.org/manual/core/authentication/#security-authentication-mechanisms if auth_mechanism is None: self.auth_mechanism = "MONGODB-CR"#by default authMechanism=MONGODB-CR , change if required else: self.auth_mechanism = auth_mechanism #Construct the connection URI "mongodb://user:password@hostnameOrIP:port/database?authMechanism=MONGODB-CR" self.uri = "mongodb://"+username+":"+password+"@"+self.host+":"+self.port+"/"+database+"?"+"authMechanism="+self.auth_mechanism self.client = pymongo.MongoClient(self.uri) self.db = None result_flag = False try: if database in self.client.database_names(): self.db = self.client[database] if self.db is not None: result_flag = True except Exception,e: print '\nException when connecting to Mongo instance' print 'PYTHON SAYS:' print e print '\n' return result_flag def connect2(self,database,username,password): "Another way to connect to the database" self.uri = self.host+":"+self.port self.client = pymongo.MongoClient(self.uri) self.db = self.client[database] result_flag = self.db.authenticate(username,password) return result_flag def insert_document(self,document,collection_name): "Insert a document into a collection" result_flag = False try: self.collection = self.get_collection(collection_name) self.collection.insert(document) except Exception,e: print '\nException in insert_document' print 'PYTHON SAYS:' print e print '\n' else: result_flag = True return result_flag def get_collection(self,collection_name): "Return the collection object with name collection_name" self.collection = None try: self.collection = self.db[collection_name] except Exception,e: print "Collection %s not found"%collection_name return self.collection def update_document(self,doc_key,doc_update,collection_name): "Update a document based on its key" result_flag = False try: self.collection = self.get_collection(collection_name) result_obj = self.collection.update(doc_key,doc_update) if result_obj['updatedExisting'] is True: result_flag = True except Exception,e: print '\nException in update_document' print 'PYTHON SAYS:' print e print '\n' return result_flag def delete_document(self,doc_key,collection_name): "Delete a given document" result_flag = False try: self.collection = self.get_collection(collection_name) result_obj = self.collection.remove(doc_key) if result_obj['n'] is 1: result_flag = True except Exception,e: print '\nException in delete_document' print 'PYTHON SAYS:' print e print '\n' else: return result_flag #---EXAMPLE USAGE--- if __name__=='__main__': 'user created with role:readWrite - Example: db.createUser({"user":"raji1","pwd":"Testdb123","roles":[{"role":"readWrite" OR "dbAdmin","db":"qxf2"}]}) ' #mongo connection string parameters (read these from config file) host = "127.0.0.1" port = "27017" user = "raji" password = "Testdb123" db_name = "qxf2" collection = "Employees" document = {"name":"Jack","address":[{"area":"NRI Layout","city":"blr","pin":"560016","state":"KA"},{"area":"NRI Layout","city":"blr","pin":"560016","state":"KA"}],"contact":"90000000006","emp_id":"0007"} document_key = {"name":"Jack"} #initialize mongo driver object mongo_obj = Mongo_Driver(host,port) #connect to a db if mongo_obj.connect(db_name,user,password) is True: print "DB authorization successful" #insert document if mongo_obj.insert_document(document,collection) is True: print "insert document successful" else: print "insert document unsuccessful" #update document if mongo_obj.update_document(document_key,document,collection) is True: print "update document successful" else: print "update document unsuccessful" #delete document if mongo_obj.delete_document(document_key,collection) is True: print "delete document successful" else: print "delete document unsuccessful" else: print "DB authorization unsuccessful" |
I hope this helps you get started with MongoDB. Questions? Comments? Post them in the comments section below and I will get back to you soon.
I have around 10 years of experience in QA , working at USA for about 3 years with different clients directly as onsite coordinator added great experience to my QA career . It was great opportunity to work with reputed clients like E*Trade financial services , Bank of Newyork Mellon , Key Bank , Thomson Reuters & Pascal metrics.
Besides to experience in functional/automation/performance/API & Database testing with good understanding on QA tools, process and methodologies, working as a QA helped me gaining domain experience in trading , banking & investments and health care.
My prior experience is with large companies like cognizant & HCL as a QA lead. I’m glad chose to work for start up because of learning curve ,flat structure that gives freedom to closely work with higher level management and an opportunity to see higher level management thinking patterns and work culture .
My strengths are analysis, have keen eye for attention to details & very fast learner . On a personal note my hobbies are listening to music , organic farming and taking care of cats & dogs.
One thought on “%1$s”