perfspec-learning/README.md

---
gitea: none
include_toc: true
---

# PerfSPEC Learning Phase

Based in [PrefSPEC: Performance Profiling-based Proactive Security Policy Enforcement for Containers](https://ieeexplore.ieee.org/document/10577533) document presented in [1], thir repository contains source files used to generate and process data.

Main Reference: [PrefSPEC document](PerfSPEC.pdf) as [White paper](https://en.wikipedia.org/wiki/White_paper) 

[Presentación in Spanish](presentacion.pdf)

[How to install](https://repo.jesusperez.pro/jesus/perfspec-learning/src/branch/main/install.md) covers basic enviroment,tools, and recommendations.

<div style="margin: auto">
 <a target="_blank" href="perfspec-learning/src/branch/main/presentacion.pdf"><img src="imgs/prefSPEC-learning.png" width="800"></a>
</div>

__PerfSPEC__ 

>[!IMPORTANT]  
With `PerfSPEC` [Security Policies](https://en.wikipedia.org/wiki/Security_policy) can be managed / watched in **Proactive** mode by using <u>ranking</u>, <u>learning</u> and <u>profiles</u> for safetiness and performance. 

It has three phases:

- Ranking
- Learning
- Runtime

This repository is focused in __Learning__ phase with attention on:

- Event logs info load and process
- Predictive learning model 

> [!NOTE]
> It is considered that __event data collection__ in `raw-audit-logs.log.xz` are realistic and representative to simulate
administrative operations.

## Files

### Data 

- `raw-audit-logs.log` contains raw Kubernetes audit logs collected using the `audit-policy.yaml` audit policy.

### Layout 

Tools are distributed in directories:

- [Collect](collect)
- [Process](process)
- [Learning](learning)

<details open>
  <summary>Files layout</summary>
    Content structure overview with notes
    <pre>
    ├── PerfSPEC.pdf                   Reference document
    ├── README.md
    ├── about.md
    ├── actions_distribution.pdf       Generated actions distribytion
    ├── collect                        Collect logs scripts 
    │   ├── audit-policy.yaml
    │   ├── collect.py
    │   └── helm-charts.json
    ├── data                           Extracted from compress archive 
    │   ├── actions-dataset-audit.txt
    │   ├── actions-logs.log
    │   ├── actions_distribution.pdf
    │   ├── main-audit-logs.log
    │   └── raw-audit-logs.log
    ├── data_sample.tar.xz             Compress archive with 'data'
    ├── imgs
    ├── install.md                     Installation notes
    ├── intro.md
    ├── learning
    │   └── python
    │       ├── __pycache__            Ignored in git
    │       ├── lib_perfspec.py
    │       ├── model_perfspec.py
    │       ├── prepare_perfspec.py
    │       ├── run_perfspec.py
    │       └── train_perfspec.py
    ├── models                        Extracted from compress archive
    │   ├── checkpoints
    │   │   ├── model_at_epoch_175.keras
    │   │   └── model_at_epoch_185.keras
    │   ├── history.json
    │   └── perfSPEC_model.keras
    ├── models_sample.tar.xz          Comperss archive with 'models'
    ├── presentacion.pdf              Presentation slides
    └── raw-audit-logs.log.xz         Main Raw Logs file
    </pre>
</details>

As some tasks can be used in [Python](https://python.org) or [Rust](https://www.rust-lang.org/) there are or will be directories for each programming languge inside directories tasks.  

Each `task/programming-language` use a common __data__ directory where processing output files is generated. 

## Collect data

If you wish to [collect](collect) your own dataset, there are several source files that might help:

- `collect/collect.py` is a script to trigger the installation and uninstallation of public Helm repositories.
- `collect/helm-charts.json` is a backup of Helm charts used at the time of the collection.

## Process data 


## Learning



## Reference

[1]: [H. Kermabon-Bobinnec et al., "PerfSPEC: Performance Profiling-based Proactive Security Policy Enforcement for Containers," in IEEE Transactions on Dependable and Secure Computing, doi: 10.1109/TDSC.2024.3420712.](https://ieeexplore.ieee.org/document/10577533)