perfspec-learning/about.md

---
gitea: none
include_toc: true
---

# [PerfSPEC Learning Phase](README.md) - ABOUT

Based in [PerfSPEC: Performance Profiling-based Proactive Security Policy Enforcement for Containers](https://ieeexplore.ieee.org/document/10577533) document presented in [1], this repository contains source files used to generate and process data.

For more [PerfSPEC](README.md) info use: 

- [Main description](README.md) 
- [Introduction](intro.md) 
- [Reference document](PerfSPEC.pdf)
- [Presentación in Spanish](presentacion.pdf)
- [How to install](https://repo.jesusperez.pro/jesus/perfspec-learning/src/branch/main/install.md)
- [Autogenerated graph view of actions and events distribution](actions_distribution.pdf)

<div style="margin: auto">
 <a target="_blank" href="perfspec-learning/src/branch/main/presentacion.pdf"><img src="imgs/perfSPEC-learning.png" width="800"></a>
</div>

# What is done so far ? 

- [X] Good look and feel and interactions among processing, analisys and presentation layers
- [X] Use better software packages management like [uv](https://docs.astral.sh/uv/) to complement Python **pip**
- [X] Use an <u>open</u> and compatible notebook like [Marimo](https://marimo.io/) to support alternative dataframes engines like [Polars](https://pola.rs/) rather than [Pandas](https://pandas.pydata.org/) 
- [X] Use settings and structures to play with different settinigs and options 
- [X] Implement one customized [LSTM](https://en.wikipedia.org/wiki/Long_short-term_memory) model within notebooks
- [X] Use of different metrics to apply to the training models with custumezable adjustments and checkpoints
- [X] Use notebooks as python scripts for command-line use cases like collect predictions or automatic train models
- [X] Use [Dry](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself) to reuse code and centralize common tasks like settings or loading resources. This is main use and existence of [lib_perfspec.py](learning/python/lib_perfspec.py)
- [X] Splitting basic tasks among several specific **notebooks**: 
    - **Preprocessing data** collection to generate clean and usefull (critical actions) info to train models and ranking [prepare_perfspec.py](learning/python/prepare_perfspec.py)
    - **Train models** to get predictions [train_perfspec.py](learning/python/train_perfspec.py)
    - **Get predictions** from existing models [run_perfspec.py](learning/python/run_perfspec.py)
    - **Review and analisys** trained models for better results [model_perfspec.py](learning/python/model_perfspec.py) 

# Plans 

## Tools and enviroment

- [ ] Use [Polars](https://pola.rs/) by default 
- [ ] Use [Nushell](https://www.nushell.sh/] for command-line interfrace and data processing
- [ ] Try [Rust Machine Learning](https://burn.dev/) as alternative or complement of [Keras](https://keras.io/) / [TensorFlow requiremens](https://www.tensorflow.org/install/pip)
- [ ] Incorporate other training models and [Large language model][LLM]

## [PerfSPEC](README.md)

- [ ] Borrow some parts to [PerfSPEC](README.md) **Ranking Phase** in classify and measure resources and performance costs
- [ ] Analisys and connection with other parts of [PerfSPEC design and reference](PerfSPEC.pdf) like **Ranking** and **Runtime** phases 

# In Review

- [ ] Check predictions accuracy 
- [ ] Validate data procedures and results
- [ ] Verify and test use cases

## Reference

[1]: [H. Kermabon-Bobinnec et al., "PerfSPEC: Performance Profiling-based Proactive Security Policy Enforcement for Containers," in IEEE Transactions on Dependable and Secure Computing, doi: 10.1109/TDSC.2024.3420712.](https://ieeexplore.ieee.org/document/10577533)