Skip to content

P
preprocess-data
Commit ea0e9b5b authored by dasharatha.vamshi's avatar dasharatha.vamshi
Browse files
Options

pickle-file

parent 5ed08153
Show whitespace changes
Inline Side-by-side
Showing with 11 additions and 1 deletion
logs/PreProccessComponent.log
View file @ ea0e9b5b
...@@ -39,3 +39,13 @@ ...@@ -39,3 +39,13 @@
2021-02-23 18:14:52,645 INFO PreProccessComponent Parsing the pickle file 2021-02-23 18:14:52,645 INFO PreProccessComponent Parsing the pickle file
2021-02-23 18:14:52,650 INFO PreProccessComponent Got the data writing it to pickle file (preprocessed_X.pkl) 2021-02-23 18:14:52,650 INFO PreProccessComponent Got the data writing it to pickle file (preprocessed_X.pkl)
2021-02-23 18:14:52,650 INFO PreProccessComponent Component executed Successfully 2021-02-23 18:14:52,650 INFO PreProccessComponent Component executed Successfully
2021-02-23 19:02:28,341 INFO PreProccessComponent Reading Json file
2021-02-23 19:02:28,351 INFO PreProccessComponent Parsing the pickle file
2021-02-23 19:02:29,640 INFO PreProccessComponent Got the data writing it to pickle file (preprocessed_X.pkl)
2021-02-23 19:02:29,641 INFO PreProccessComponent Component executed Successfully
2021-02-23 19:03:02,447 INFO PreProccessComponent Reading Json file
2021-02-23 19:03:02,457 INFO PreProccessComponent Parsing the pickle file
2021-02-23 19:03:14,509 INFO PreProccessComponent Reading Json file
2021-02-23 19:03:14,519 INFO PreProccessComponent Parsing the pickle file
2021-02-23 19:03:15,704 INFO PreProccessComponent Got the data writing it to pickle file (preprocessed_X.pkl)
2021-02-23 19:03:15,705 INFO PreProccessComponent Component executed Successfully
scripts/common/config_parser.py
View file @ ea0e9b5b
...@@ -35,7 +35,7 @@ config = { ...@@ -35,7 +35,7 @@ config = {
"shared_volume": os.environ.get("shared_volume"), "shared_volume": os.environ.get("shared_volume"),
"json_path": os.environ.get("json_path"), "json_path": os.environ.get("json_path"),
"fillna_method": os.environ.get("fillna_method", default="ffill"), "fillna_method": os.environ.get("fillna_method", default="ffill"),
"standard_scalar_path": os.environ.get("standard_scalar_path"), "standard_scalar_path": os.environ.get("standard_scalar_path",default=r"scripts\common\StandardScaler.pkl"),
} }
if not os.path.exists(config['shared_volume']): if not os.path.exists(config['shared_volume']):
sys.stderr.write("Shared path does not exist!") sys.stderr.write("Shared path does not exist!")
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment