AI-Based Cyber Threats Detection System

The problem is to identify suspicious activity from diverse security events while keeping false alarms manageable and providing enough context for a human analyst to validate each alert. 1.2 Objectives

• Design a modular pipeline for authorised log and network-flow analysis.
• Prepare data consistently and prevent training-test leakage.
• Compare a baseline classifier with an anomaly-detection approach.
• Generate risk-ranked alerts with understandable feature-level context.
• Evaluate the design with standard detection metrics, not accuracy alone. 1.3 Scope The system is intended for defensive monitoring in a lab or organisation that has permission to analyse its logs. It does not replace endpoint protection, firewalls, incident-response procedures, or expert investigation.

Research in intrusion detection has explored statistical analysis, rule-based systems, classical machine learning, deep learning, and hybrid approaches. NSL-KDD has been used for reproducible intrusion-detection experiments, although it does not fully represent current enterprise traffic. CICIDS2017 provides labelled flows for benign traffic and multiple attack scenarios, but evaluation must account for dataset bias, class imbalance, and collection assumptions. Sommer and Paxson discuss the practical difficulty of applying machine learning to network intrusion detection when operational conditions differ from training data. Buczak and Guven survey data -mining and machine-learning methods for cybersecurity analytics and note that method choice depends on data, attack type, and operational constraints. These studies motivate careful validation, explainability, and ongoing monitoring rather than relying on benchmark scores alone.

High scores on public benchmarks do not guarantee reliable live-network performance. A practical design should document data provenance, use leakage-resistant validation, report false-positive rates, include analyst review, and monitor drift after deployment.

The system accepts authorised network-flow records and security events, transforms them into a common feature representation, and passes them to a trained classifier and an anomaly detector. Outputs are combined into a risk score. Aler ts are stored with timestamps, model version, relevant features, and review status so an analyst can confirm, dismiss, or escalate them.

Figure 1. Proposed processing flow (conceptual): Authorised Logs → Validation & Preprocessing → Feature Extraction → ML Classifier + Anomaly Detector → Risk Scoring → Alert Dashboard → Analyst Review

A prototype can use Python for data processing and inference, scikit-learn for baseline algorithms, FastAPI for a REST API, PostgreSQL or SQLite for alert metadata, and a React dashboard. A local CSV replay pipeline is sufficient for an academic prototype; live collection should only be added with explicit authorisation and access controls. 4. Methodology

The prototype may begin with a public labelled dataset such as CICIDS2017 or NSL -KDD after checking the dataset documentation and terms. Local logs should only be analysed with permission, and personally identifiable information should be minimised or pseudonymised. This paper does not claim that experiments have already been completed. 4.2 Preprocessing

• Inspect field names, data types, missing values, duplicates, and label distribution.
• Remove identifiers that could leak labels or cause memorisation of dataset-specific records.
• Split by time, host, or capture session where possible to reduce near-duplicate leakage.
• Fit encoders and scalers on training data only and reuse the saved transformations for testing.
• Handle class imbalance with class weights or training-only resampling; do not resample the test set.

A Random Forest or gradient-boosted tree model can serve as a supervised baseline for tabular features. An Isolation Forest can provide an anomaly score when labels are incomplete. Outputs should be calibrated on validation data and combined conservatively; a risk score must not be presented as certainty.

Each alert should include a timestamp, source-record identifier, predicted category, confidence or anomaly score, severity band, and concise explanation. Thresholds should be selected using validation data and operational tolerance for false positives. High-impact responses should require human confirmation; the prototype should not automatically disable accounts or block traffic.

1. Load authorised records and validate the schema. 2. Apply saved preprocessing. 3. Obtain classifier probabilities and anomaly scores. 4. Combine scores with transparent rules. 5. Create alerts above a validated threshold. 6. Store evidence and model version. 7. Allow analyst review. 8. Periodically evaluate drift and retrain only after approval.

Authenticate API users and apply role-based access control.

• Encrypt traffic in transit and restrict access to stored logs.
• Keep secrets outside source code using environment variables or a secret manager.
• Record administrative actions and model-version changes in an audit log.
• Validate input schemas and limit upload sizes and request rates.
• Set retention limits and remove data that is no longer required.

No numeric results are reported because no experiment has been conducted for this manuscript. A completed implementation should publish the dataset split, preprocessing steps, model parameters, software versions, confusion matrix, and per -class metrics to support reproducibility.

7.1 Advantages

• Combines known-pattern classification with anomaly detection. • Can begin with offline datasets and be extended modularly. • Supports analyst review and traceable alert metadata. • Uses standard metrics to make operational trade-offs visible. 7.2 Limitations • Model quality depends on relevance and quality of training data. • False positives can overwhelm analysts if thresholds are poorly chosen. • Changing traffic can cause model drift. • Public benchmark performance may not generalise to a real organisation. • Adversarial evasion and encrypted traffic may limit available features.

Use the system only on networks and logs for which permission has been granted. Monitoring should follow applicable law and organisational policy. Data minimisation, retention controls, restricted access, and careful handling of identifiers are essential. Model outputs should support rather than replace human judgement, and responses must follow an approved incident-response process.

Future work may add streaming ingestion, threat-intelligence enrichment, explainability methods such as SHAP for supported models, drift detection, privacy-preserving learning, and integration with a security information and event management (SIEM) platform. Validation on temporally separated and organisation-specific data is required before production use. 10. Conclusion This paper proposed an AI-Based Cyber Threats Detection System combining supervised classification, anomaly detection, risk scoring, and analyst-reviewed alerts. It emphasises sound data preparation, leakage-resistant evaluation, understandable alert context, and privacy-conscious deployment. The design can support a defensive academic prototype, but its practical value must be established through reproducible experiments and controlled validation.