62 lines
3.6 KiB
Markdown
62 lines
3.6 KiB
Markdown
---
|
|
gitea: none
|
|
include_toc: true
|
|
---
|
|
|
|
# [PerfSPEC Learning Phase](README.md) - ABOUT
|
|
|
|
Based in [PerfSPEC: Performance Profiling-based Proactive Security Policy Enforcement for Containers](https://ieeexplore.ieee.org/document/10577533) document presented in [1], this repository contains source files used to generate and process data.
|
|
|
|
For more [PerfSPEC](README.md) info use:
|
|
|
|
- [Main description](README.md)
|
|
- [Introduction](intro.md)
|
|
- [Reference document](PerfSPEC.pdf)
|
|
- [Presentación in Spanish](presentacion.pdf)
|
|
- [How to install](https://repo.jesusperez.pro/jesus/perfspec-learning/src/branch/main/install.md)
|
|
- [Autogenerated graph view of actions and events distribution](actions_distribution.pdf)
|
|
|
|
<div style="margin: auto">
|
|
<a target="_blank" href="perfspec-learning/src/branch/main/presentacion.pdf"><img src="imgs/perfSPEC-learning.png" width="800"></a>
|
|
</div>
|
|
|
|
# What is done so far ?
|
|
|
|
- [X] Good look and feel and interactions among processing, analisys and presentation layers
|
|
- [X] Use better software packages management like [uv](https://docs.astral.sh/uv/) to complement Python **pip**
|
|
- [X] Use an <u>open</u> and compatible notebook like [Marimo](https://marimo.io/) to support alternative dataframes engines like [Polars](https://pola.rs/) rather than [Pandas](https://pandas.pydata.org/)
|
|
- [X] Use settings and structures to play with different settinigs and options
|
|
- [X] Implement one customized [LSTM](https://en.wikipedia.org/wiki/Long_short-term_memory) model within notebooks
|
|
- [X] Use of different metrics to apply to the training models with custumezable adjustments and checkpoints
|
|
- [X] Use notebooks as python scripts for command-line use cases like collect predictions or automatic train models
|
|
- [X] Use [Dry](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself) to reuse code and centralize common tasks like settings or loading resources. This is main use and existence of [lib_perfspec.py](learning/python/lib_perfspec.py)
|
|
- [X] Splitting basic tasks among several specific **notebooks**:
|
|
- **Preprocessing data** collection to generate clean and usefull (critical actions) info to train models and ranking [prepare_perfspec.py](learning/python/prepare_perfspec.py)
|
|
- **Train models** to get predictions [train_perfspec.py](learning/python/train_perfspec.py)
|
|
- **Get predictions** from existing models [run_perfspec.py](learning/python/run_perfspec.py)
|
|
- **Review and analisys** trained models for better results [model_perfspec.py](learning/python/model_perfspec.py)
|
|
|
|
# Plans
|
|
|
|
## Tools and enviroment
|
|
|
|
- [ ] Use [Polars](https://pola.rs/) by default
|
|
- [ ] Use [Nushell](https://www.nushell.sh/] for command-line interfrace and data processing
|
|
- [ ] Try [Rust Machine Learning](https://burn.dev/) as alternative or complement of [Keras](https://keras.io/) / [TensorFlow requiremens](https://www.tensorflow.org/install/pip)
|
|
- [ ] Incorporate other training models and [Large language model][LLM]
|
|
|
|
## [PerfSPEC](README.md)
|
|
|
|
- [ ] Borrow some parts to [PerfSPEC](README.md) **Ranking Phase** in classify and measure resources and performance costs
|
|
- [ ] Analisys and connection with other parts of [PerfSPEC design and reference](PerfSPEC.pdf) like **Ranking** and **Runtime** phases
|
|
|
|
# In Review
|
|
|
|
- [ ] Check predictions accuracy
|
|
- [ ] Validate data procedures and results
|
|
- [ ] Verify and test use cases
|
|
|
|
## Reference
|
|
|
|
[1]: [H. Kermabon-Bobinnec et al., "PerfSPEC: Performance Profiling-based Proactive Security Policy Enforcement for Containers," in IEEE Transactions on Dependable and Secure Computing, doi: 10.1109/TDSC.2024.3420712.](https://ieeexplore.ieee.org/document/10577533)
|