Upload New File

assignments_assignment1__1_.md

+
+# Assignment 1
+
+Hey there! Welcome to Knowledge Lens Intern Training Program.
+
+This Assignment will serve as a quick refresher on the usage of NoSQL and Time-series databases.
+There are three tasks in this assignment, on completion of which you'll learn:
+*  How to interact with Mongo DB
+*  Using Pandas Dataframe and generating your own excel reports
+*  Leveraging Kairos Time-series database for data ingestion and querying the same
+*  Publishing and Consuming messages via MQTT protocol
+*  Caching mechanism using Redis DB 
+
+Happy Coding! :tada:
+
+## :pushpin: Task 1: Working with Mongo - Advanced
+
+
+### :golf: Areas covered:
+- Timeseries Operation
+- Working with NoSQL
+- Working with Pandas
+
+### :books: Description:
+You are given with a dataset of an inspection company in the form of a JSON file. The end goal of the project is to create an API interface that will provide the following: 
+
+1. Business name with maximum number of "`Violation Issued`".
+2. Business name that has no violation.
+3. Generate Excel Report based on `result`, `business name` and `date`
+
+Sample Document:
+```json
+{
+  "id": "10021-2015-ENFO",
+  "certificate_number": 9278806,
+  "business_name": "ATLIXCO DELI GROCERY INC.",
+  "date": "Feb 20 2015",
+  "result": "No Violation Issued",
+  "sector": "Cigarette Retail Dealer - 127",
+  "address": {
+    "city": "RIDGEWOOD",
+    "zip": 11385,
+    "street": "MENAHAN ST",
+    "number": 1712
+  }
+}
+```
+
+Bonus Points: Use Mongo Aggregate framework
+
+### :wrench: Tools to use: 
+1. Pycharm / VSCode
+2. Robo3T / Studio3T / MongoDB Compass
+3. PyMongo
+
+
+### :mag: References:
+* [Querying Documents on Mongo](https://www.mongodb.com/docs/manual/tutorial/query-documents/)
+* [Quick Summary on Mongo Aggregation Stages](https://www.mongodb.com/docs/manual/reference/operator/aggregation-pipeline/)
+* [Generating Excel Sheets from a Pandas Dataframe](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_excel.html)
+* [How to return files on FastAPI response](https://fastapi.tiangolo.com/advanced/custom-response/#fileresponse)
+* [PyMongo Official Documentation](https://pymongo.readthedocs.io/en/stable/)
+
+
+
+_________________________________
+
+## :pushpin: Task 2: Working with Timeseries
+
+
+### :golf: Areas covered:
+- Timeseries Operation
+- Working with Timeseries
+- Working with Pandas
+
+### :books: Description:
+You are given with a dataset of weather in the form of a CSV file. The end goal of the project is to create an API interface that will provide the following: 
+
+1. Get daily, monthly, weekly and monthly aggregate (min, max, and average) of the data and generate report in Excel format.
+
+Sample Document:
+
+|Date                         |Time         |CO(GT)|PT08.S1(CO)|NMHC(GT)   |C6H6(GT)|PT08.S2(NMHC)|NOx(GT)|PT08.S3(NOx)|NO2(GT)|PT08.S4(NO2)|PT08.S5(O3)|T   |RH  |AH    |
+|-----------------------------|-------------|------|-----------|-----------|--------|-------------|-------|------------|-------|------------|-----------|----|----|------|
+|10/03/2004                   |18:00:00     |2.6   |1360       |150        |11.9    |1046         |166    |1056        |113    |1692        |1268       |13.6|48.9|0.7578|
+
+
+### :wrench: Tools to use: 
+1. Pycharm / VSCode
+2. Pandas 
+3. Kairos
+
+### :mag: References:
+* [How to query Kairos DB using Metrics](https://kairosdb.github.io/docs/restapi/QueryMetrics.html)
+
+
+------------------------------------------------------
+
+## :pushpin: Task 3: Working with MQTT & REDIS
+
+### :golf: Areas covered:
+- MQTT Protocol
+- Caching using Redis DB
+
+### :books: Description
+
+Hospital with 'N' doctor's on duty,
+Receptionist need's to assign new patient's in round robin order. 
+Patient treated by a doctor should be sent to same doctor in his next visit.
+   
+   
+Consider N doctors as N Mqtt Topics,
+
+Patient details sample format: 
+```json
+  {
+	   "patient_id" : "patient_1",
+	   "description" : "Fever",
+	   "age" : 22,
+	   "name": "Raju"	   
+	   }
+```
+	   
+Use Redis for caching information
+
+Create a snippet which produces patient information to MQTT.
+
+Create consumer's which consumes data from these topics and store to a redis db
+
+### :wrench: Tools to use: 
+1. Pycharm / VSCode
+2. MQTT - (PIP package: `paho-mqtt`)
+3. REDIS - (PIP package: `redis`)
+
+### :mag: References:
+* [Using MQTT in Python](https://www.emqx.com/en/blog/how-to-use-mqtt-in-python)
+* [Connecting to Redis in Python](https://docs.redis.com/latest/rs/references/client_references/client_python/)