llmsecops-research/infrastructure

Infrastructure

This directory exists to contain the foundation model (pre-trained generative language model). The foundation model files are ignored by git.

Model Choice

The foundation model for this project needed to work under multiple constraints:

1. Repo storage limits: Even with Git LFS enabled, GitHub restricts repository size to 5GB (at least for the free tier).
2. Build system storage limits: Standard Linux runners in GitHub Actions have a 16GB SSD.

The CPU-optimized microsoft/Phi-3-mini-4k-instruct-onnx model met this storage space requirement.

Provisioning the Foundation Model

The foundation model dependency is loaded differently for local development vs. the build system:

1. Local: The model is downloaded once by the ./run.sh shell script at the project root, but excluded in .gitignore since it's too large for GitHub's LFS limitations.
2. Build System: The model is downloaded on every workflow run with huggingface-cli.