perfspec-learning/README.md

Table of Contents

• PerfSPEC Learning Phase
• Files
• Data
• Layout
• Collect data
• Process data
• Learning
• Reference

PerfSPEC Learning Phase

Based in PrefSPEC: Performance Profiling-based Proactive Security Policy Enforcement for Containers document presented in [1], thir repository contains source files used to generate and process data.

Main Reference: PrefSPEC document as White paper

Presentación in Spanish

How to install covers basic enviroment,tools, and recommendations.

PerfSPEC

Important

With PerfSPEC Security Policies can be managed / watched in Proactive mode by using ranking, learning and profiles for safetiness and performance.

It has three phases:

• Ranking
• Learning
• Runtime

This repository is focused in Learning phase with attention on:

• Event logs info load and process
• Predictive learning model

Note

It is considered that event data collection in raw-audit-logs.log.xz are realistic and representative to simulate administrative operations.

Files

Data

• raw-audit-logs.log contains raw Kubernetes audit logs collected using the audit-policy.yaml audit policy.

Layout

Tools are distributed in directories:

• Collect
• Process
• Learning

Files layout

Content structure overview with notes

    ├── PerfSPEC.pdf                   Reference document
    ├── README.md
    ├── about.md
    ├── actions_distribution.pdf       Generated actions distribytion
    ├── collect                        Collect logs scripts 
    │   ├── audit-policy.yaml
    │   ├── collect.py
    │   └── helm-charts.json
    ├── data                           Extracted from compress archive 
    │   ├── actions-dataset-audit.txt
    │   ├── actions-logs.log
    │   ├── actions_distribution.pdf
    │   ├── main-audit-logs.log
    │   └── raw-audit-logs.log
    ├── data_sample.tar.xz             Compress archive with 'data'
    ├── imgs
    ├── install.md                     Installation notes
    ├── intro.md
    ├── learning
    │   └── python
    │       ├── __pycache__            Ignored in git
    │       ├── lib_perfspec.py
    │       ├── model_perfspec.py
    │       ├── prepare_perfspec.py
    │       ├── run_perfspec.py
    │       └── train_perfspec.py
    ├── models                        Extracted from compress archive
    │   ├── checkpoints
    │   │   ├── model_at_epoch_175.keras
    │   │   └── model_at_epoch_185.keras
    │   ├── history.json
    │   └── perfSPEC_model.keras
    ├── models_sample.tar.xz          Comperss archive with 'models'
    ├── presentacion.pdf              Presentation slides
    └── raw-audit-logs.log.xz         Main Raw Logs file
    

As some tasks can be used in Python or Rust there are or will be directories for each programming languge inside directories tasks.

Each task/programming-language use a common data directory where processing output files is generated.

Collect data

If you wish to collect your own dataset, there are several source files that might help:

• collect/collect.py is a script to trigger the installation and uninstallation of public Helm repositories.
• collect/helm-charts.json is a backup of Helm charts used at the time of the collection.

Process data

Learning

Reference

[1]: H. Kermabon-Bobinnec et al., "PerfSPEC: Performance Profiling-based Proactive Security Policy Enforcement for Containers," in IEEE Transactions on Dependable and Secure Computing, doi: 10.1109/TDSC.2024.3420712.