
# Assignment 1

Hey there! Welcome to Knowledge Lens Intern Training Program.

This Assignment will serve as a quick refresher on the usage of NoSQL and Time-series databases.
There are three tasks in this assignment, on completion of which you'll learn:
*  How to interact with Mongo DB
*  Using Pandas Dataframe and generating your own excel reports
*  Leveraging Kairos Time-series database for data ingestion and querying the same
*  Publishing and Consuming messages via MQTT protocol
*  Caching mechanism using Redis DB 

Happy Coding! :tada:

## :pushpin: Task 1: Working with Mongo - Advanced


### :golf: Areas covered:
- Timeseries Operation
- Working with NoSQL
- Working with Pandas

### :books: Description:
You are given with a dataset of an inspection company in the form of a JSON file. The end goal of the project is to create an API interface that will provide the following: 

1. Business name with maximum number of "`Violation Issued`".
2. Business name that has no violation.
3. Generate Excel Report based on `result`, `business name` and `date`

Sample Document:
```json
{
  "id": "10021-2015-ENFO",
  "certificate_number": 9278806,
  "business_name": "ATLIXCO DELI GROCERY INC.",
  "date": "Feb 20 2015",
  "result": "No Violation Issued",
  "sector": "Cigarette Retail Dealer - 127",
  "address": {
    "city": "RIDGEWOOD",
    "zip": 11385,
    "street": "MENAHAN ST",
    "number": 1712
  }
}
```

Bonus Points: Use Mongo Aggregate framework

### :wrench: Tools to use: 
1. Pycharm / VSCode
2. Robo3T / Studio3T / MongoDB Compass
3. PyMongo


### :mag: References:
* [Querying Documents on Mongo](https://www.mongodb.com/docs/manual/tutorial/query-documents/)
* [Quick Summary on Mongo Aggregation Stages](https://www.mongodb.com/docs/manual/reference/operator/aggregation-pipeline/)
* [Generating Excel Sheets from a Pandas Dataframe](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_excel.html)
* [How to return files on FastAPI response](https://fastapi.tiangolo.com/advanced/custom-response/#fileresponse)
* [PyMongo Official Documentation](https://pymongo.readthedocs.io/en/stable/)



_________________________________

## :pushpin: Task 2: Working with Timeseries


### :golf: Areas covered:
- Timeseries Operation
- Working with Timeseries
- Working with Pandas

### :books: Description:
You are given with a dataset of weather in the form of a CSV file. The end goal of the project is to create an API interface that will provide the following: 

1. Get daily, monthly, weekly and monthly aggregate (min, max, and average) of the data and generate report in Excel format.

Sample Document:

|Date                         |Time         |CO(GT)|PT08.S1(CO)|NMHC(GT)   |C6H6(GT)|PT08.S2(NMHC)|NOx(GT)|PT08.S3(NOx)|NO2(GT)|PT08.S4(NO2)|PT08.S5(O3)|T   |RH  |AH    |
|-----------------------------|-------------|------|-----------|-----------|--------|-------------|-------|------------|-------|------------|-----------|----|----|------|
|10/03/2004                   |18:00:00     |2.6   |1360       |150        |11.9    |1046         |166    |1056        |113    |1692        |1268       |13.6|48.9|0.7578|


### :wrench: Tools to use: 
1. Pycharm / VSCode
2. Pandas 
3. Kairos

### :mag: References:
* [How to query Kairos DB using Metrics](https://kairosdb.github.io/docs/restapi/QueryMetrics.html)


------------------------------------------------------

## :pushpin: Task 3: Working with MQTT & REDIS

### :golf: Areas covered:
- MQTT Protocol
- Caching using Redis DB

### :books: Description

Hospital with 'N' doctor's on duty,
Receptionist need's to assign new patient's in round robin order. 
Patient treated by a doctor should be sent to same doctor in his next visit.
   
   
Consider N doctors as N Mqtt Topics,

Patient details sample format: 
```json
  {
	   "patient_id" : "patient_1",
	   "description" : "Fever",
	   "age" : 22,
	   "name": "Raju"	   
	   }
```
	   
Use Redis for caching information

Create a snippet which produces patient information to MQTT.

Create consumer's which consumes data from these topics and store to a redis db

### :wrench: Tools to use: 
1. Pycharm / VSCode
2. MQTT - (PIP package: `paho-mqtt`)
3. REDIS - (PIP package: `redis`)

### :mag: References:
* [Using MQTT in Python](https://www.emqx.com/en/blog/how-to-use-mqtt-in-python)
* [Connecting to Redis in Python](https://docs.redis.com/latest/rs/references/client_references/client_python/)
