Commit 3f0cd93
Implement serialization of Summarizer (#1293)
Summary:
To save a checkpoint of a Feature Importance job, we will need to persistent below fields in https://www.internalfb.com/code/fbsource/[5e3ebcd68e0c]/fbcode/model_understanding/feature_importance/summarizer.py?lines=89-99 in order save the state of the job:
```
# Initialize summarizers
self.input_attr_summarizer = Summarizer([Mean(), StdDev(order=0)])
self.input_coverage_summarizer = Summarizer([Mean()])
self.derived_attr_summarizer = Summarizer([Mean(), StdDev(order=0)])
self.derived_coverage_summarizer = Summarizer([Mean()])
self.neuron_attr_summarizer = Summarizer([Mean(), StdDev(order=0)])
# Initialize batch counts
self.input_batch_count = 0
self.derived_batch_count = 0
self.neuron_batch_count = 0
```
This diff implements (de)serialization of Summarizer.
### future diffs include:
- save FI checkpoints to manifold
- add the hook to Mast graceful preemption to save the checkpoint of Feature Importance Job
- load checkpoint when initializing the Feature Importance Job
Pull Request resolved: #1293
Reviewed By: MarcioPorto
Differential Revision: D58227186
Pulled By: lurunming
fbshipit-source-id: e99029548eef7efe37fe4e39cbbe14acadbb240d1 parent 5022a41 commit 3f0cd93
1 file changed
+2
-1
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
75 | 75 | | |
76 | 76 | | |
77 | 77 | | |
78 | | - | |
| 78 | + | |
79 | 79 | | |
80 | 80 | | |
81 | 81 | | |
| |||
147 | 147 | | |
148 | 148 | | |
149 | 149 | | |
| 150 | + | |
150 | 151 | | |
0 commit comments