jesus/perfspec-learning
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore: add and fix text

Browse Source
This commit is contained in:
Jesús Pérez Lorenzo 2025-01-27 19:24:19 +00:00
parent e5b9f98637
commit e572ab8dc5
2 changed files with 17 additions and 14 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
README.md
View File

@ -7,11 +7,9 @@ include_toc: true
Based in [PrefSPEC: Performance Profiling-based Proactive Security Policy Enforcement for Containers](https://ieeexplore.ieee.org/document/10577533) document presented in [1], this repository contains source files used to generate and process data.
Main Reference: [PrefSPEC document](PerfSPEC.pdf) as [White paper](https://en.wikipedia.org/wiki/White_paper)
[Presentación in Spanish](presentacion.pdf)
[How to install](https://repo.jesusperez.pro/jesus/perfspec-learning/src/branch/main/install.md) covers basic enviroment,tools, and recommendations.
- Main Reference: [PrefSPEC reference document](PerfSPEC.pdf) as [White paper](https://en.wikipedia.org/wiki/White_paper)
- [Presentación in Spanish](presentacion.pdf)
- [How to install](https://repo.jesusperez.pro/jesus/perfspec-learning/src/branch/main/install.md) covers basic enviroment,tools, and recommendations.
<div style="margin: auto">
<a target="_blank" href="perfspec-learning/src/branch/main/presentacion.pdf"><img src="imgs/prefSPEC-learning.png" width="800"></a>
@ -38,7 +36,7 @@ There are additional documents to this:
- [Quick start](installation.md) and installation
- [Intro](intro.md) about why and what is done
- [About](about.md) goals and experiences
- [Presentation](presentacion.pdf) slides to explain process and enviroment
- [Presentation in Spanish](presentacion.pdf) slides to explain process and enviroment
> [!NOTE]
> It is considered that __event data collection__ in `raw-audit-logs.log.xz` are realistic and representative to simulate
@ -97,7 +95,7 @@ If you wish to [collect](collect) your own dataset, there are several source fil
`data/main-audit-logs.log` Data logs fixed and clean
`data/actions-dataset-audit.txt` Source content for learning models
`data/actions_distribution.pdf` Generated graph view of actions / events distribution
`data/actions_distribution.pdf` Autogenerated graph view of actions and events distribution
## Data Models

19
about.md
View File

@ -7,9 +7,14 @@ include_toc: true
Based in [PrefSPEC: Performance Profiling-based Proactive Security Policy Enforcement for Containers](https://ieeexplore.ieee.org/document/10577533) document presented in [1], this repository contains source files used to generate and process data.
[PrefSPEC document](PerfSPEC.pdf)
For more info use:
[Presentación in Spanish](presentacion.pdf)
- [PrefSPEC main description](Readme.md)
- [PrefSPEC introduction](intro.md)
- [PrefSPEC reference document](PerfSPEC.pdf)
- [Presentación in Spanish](presentacion.pdf)
- [How to install](https://repo.jesusperez.pro/jesus/perfspec-learning/src/branch/main/install.md)
- [Autogenerated graph view of actions and events distribution](actions_distribution.pdf)
<div style="margin: auto">
<a target="_blank" href="perfspec-learning/src/branch/main/presentacion.pdf"><img src="imgs/prefSPEC-learning.png" width="800"></a>
@ -18,14 +23,14 @@ Based in [PrefSPEC: Performance Profiling-based Proactive Security Policy Enforc
# What is done so far ?
- [X] Good look and feel and interactions among processing, analisys and presentation layers
- [X] Using better software packages management like [uv](https://docs.astral.sh/uv/) to complement Python **pip**
- [X] A notebook open like [Marimo](https://marimo.io/) to support alternative dataframes engines like [Polars](https://pola.rs/) rather than [Pandas](https://pandas.pydata.org/)
- [X] Use better software packages management like [uv](https://docs.astral.sh/uv/) to complement Python **pip**
- [X] Use an <u>>open</u> and compatible notebook like [Marimo](https://marimo.io/) to support alternative dataframes engines like [Polars](https://pola.rs/) rather than [Pandas](https://pandas.pydata.org/)
- [X] Use settings and structures to play with different settinigs and options
- [X] Implement a customize one [LSTM](https://en.wikipedia.org/wiki/Long_short-term_memory) model within notebooks
- [X] Use of different metrics to apply to the training models with custumezable adjustments
- [X] Implement one customized [LSTM](https://en.wikipedia.org/wiki/Long_short-term_memory) model within notebooks
- [X] Use of different metrics to apply to the training models with custumezable adjustments and checkpoints
- [X] Use notebooks as python scripts for command-line use cases like collect predictions or automatic train models
- [X] Use [Dry](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself) to reuse code and centralize common tasks like settings or loading resources. This is main use and existence of [lib_perfspec.py](learning/python/lib_perfspec.py)
- [X] Spliting basic tasks among seveal specific **notebooks**:
- [X] Splitting basic tasks among several specific **notebooks**:
- **Preprocessing data** collection to generate clean and usefull (critical actions) info to train models and ranking [prepare_perfspec.py](learning/python/prepare_perfspec.py)
- **Train models** to get predictions [train_perfspec.py](learning/python/train_perfspec.py)
- **Get predictions** from existing models [run_perfspec.py](learning/python/run_perfspec.py)