# Assignment 4



## Task 1: Working with Mongo - Advanced


## Areas covered:
- Working with NoSQL
- Working with Pandas

## Description:
You are given with a semester deatils in a JSON format, write FAST APIs for the below  : 

1. To accept the semester details JSON and insert it in a collection
2. To get sum and average of all marks filerter by any of "student_id", "batch_id", "semster_id", "subject_id"
3. Generate excel report with "Student name", "Sememter name", "Subject 1", "Subject 2" ...., "Total"

Sample Document:
```json
{
  "semester_name": "Semester 3",
  "semester_id": 299,
  "semester_start_date": "2019-05-18",
  "semester_end_date": "2019-06-04",
  "student_details":[{
  "student_name": "Student 1",
  "student_id": 14500,
  "batch_id": 43,
  "subject_details": [
    {
      "subject_name": "Subject 1",
      "subject_id": 12,
      "score": 74
    },
    {
      "subject_name": "Subject 2",
      "subject_id": 14,
      "score": 88
    },
    {
      "subject_name": "Subject 3",
      "subject_id": 15,
      "score": 75
    },
    {
      "subject_name": "Subject 4",
      "subject_id": 5,
      "score": 85
    }
  ]
},
{
  "student_name": "Student 2",
  "student_id": 14523,
  "batch_id": 44,
  "subject_details": [
    {
      "subject_name": "Subject 1",
      "subject_id": 12,
      "score": 83
    },
    {
      "subject_name": "Subject 2",
      "subject_id": 14,
      "score": 98
    },
    {
      "subject_name": "Subject 3",
      "subject_id": 15,
      "score": 68
    },
    {
      "subject_name": "Subject 4",
      "subject_id": 5,
      "score": 71
    }
  ]
}]
}
```

Note: Perform all filter operations on Mongo itself

### Tools to use: 
1. Pycharm / VSCode
2. Robo3T / Studio3T / MongoDB Compass
3. PyMongo


### Reference:
https://www.mongodb.com/docs/manual/tutorial/query-documents/
https://www.mongodb.com/docs/manual/reference/operator/aggregation-pipeline/
https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_excel.html
https://fastapi.tiangolo.com/advanced/custom-response/#fileresponse
https://pymongo.readthedocs.io/en/stable/


________________________________________________________
## Task 2: Working with Timeseries


## Areas covered:
- Timeseries Operation
- Working with Timeseries
- Working with Pandas

## Description:
You are given with a dataset of temperature in the form of a CSV file. The end goal of the project is to create an API interface that will provide the following: 

1. Get daily, weekly and monthly aggregate (min, max, and average) of the data filter by "good" data points and generate report in Excel format.

Sample Document:



| Datetime | Temperature | Data quality |
| -------- | ----------- | ------------ |
|2022-04-03 00:01:00.000 | 112.4 | good |
|2022-04-03 00:02:00.000 | 111.32| good |
|2022-04-03 00:03:00.000 | 114.98| bad  |

### Tools to use: 
1. Pycharm / VSCode
2. Pandas 
3. Kairosdb

### Reference:
https://kairosdb.github.io/docs/restapi/QueryMetrics.html
https://pypi.org/project/kairosdb-python/

________________________________________________________
## Task 3: Working with MQTT


## Areas covered:
- MQTT operation

## Description:
For the above (task 2) given sample 

1. Push a message to the MQTT for every 5th successful record with "good" 
2. Message contains stats (sum, average,  timestamp - latest message) of all above 5 good records
3. store each aggregation to a separate redis database
4. develop an API to fetch the above saved data from redis
