Abstract
Log file analysis using large language models provides mechanisms for discovering embeddings that distinguish between different behaviors present in system logs. This work investigates unsupervised learning approaches for discriminating between normal and anomalous behaviors.
Key Contributions
- Unsupervised Anomaly Detection: We explore how LLM-derived embeddings can be used for unsupervised anomaly detection without requiring labeled training data.
- Feature Learning Analysis: We analyze how different LLM architectures learn features relevant to security log analysis.
- Practical Guidelines: We provide recommendations for practitioners deploying unsupervised log analysis systems.
Methodology
- Extract embeddings from pre-trained and fine-tuned LLMs
- Apply clustering algorithms to identify behavioral patterns
- Evaluate anomaly detection performance across multiple datasets
Key Findings
LLM embeddings capture semantic relationships in log data that enable effective unsupervised learning. The learned representations naturally separate normal operational patterns from anomalous events without explicit supervision.
Applications
This approach is particularly valuable in scenarios where labeled anomaly data is scarce or unavailable, which is common in real-world security operations.