2.1 Network Security Data Acquisition and Storage Module Design

In the design of network security situational awareness platform architecture, we need to comprehensively consider data collection, storage, processing, analysis, and threat early warning and defense. First of all, data collection is the foundation of the platform, which needs to design efficient data collection schemes to collect various types of data as comprehensively as possible ^[4]. Data sources include network traffic, logs, events, etc., and the collection methods can use network crawler, API interface, log rotation, etc. Data storage is one of the key links of the platform, which needs to design a reasonable storage architecture, including distributed storage, relational database, NoSQL database, etc. The choice of storage architecture needs to be weighed against actual requirements to ensure high data availability, high performance, and scalability.

Data processing serves as a pivotal component of the platform, encompassing both real-time and offline processing of the gathered data. Real-time processing comprises data flow management and instantaneous analysis, while offline processing involves batch processing and data mining. This process utilizes a diverse array of data processing techniques and algorithms. Analysis methods include statistical analysis, association analysis, trend prediction, etc. During the analysis process, various data analysis and mining tools, such as Tableau, Elasticsearch, can be utilized to improve efficiency of analysis ^[5]. It is also necessary to consider the output and presentation of the analysis results in order to better support the development of security decisions and defense measures. Threat early warning and defense is one of the important functions of the platform, which requires real-time early warning and defense according to the analysis results. Integration and linkage mechanisms with security equipment and defense systems need to be considered to achieve rapid response and disposal. Fig. 1 shows the network security module design.

Fig. 1. Design of security storage module.

The visual interface requires an intuitive and easy to use interface providing various functional modules and components, such as instrument boards, reports, charts, etc. The interface needs to support a variety of terminal devices, including PC, mobile phone, tablet computer, etc ^[6]. And support personalized customization and expansion to meet the needs of different users. In terms of security, data transmission and storage need to be encrypted and protected. At the same time, it is necessary to conduct regular security audit and vulnerability scanning to find and deal with security problems in time ^[7]. The design of network security situational platform architecture needs to fully consider the requirements of availability, scalability, maintainability and security.

2.2 Design of Network Security Data Processing and Analysis Module

Network security data processing undertakes the task of real-time processing and in-depth analysis of large-scale network data. The module adopts efficient data processing algorithm and advanced data analysis technology to realize accurate and real-time security threat detection and early warning ^[8].

The module supports both real-time flow processing and offline batch processing modes, tailored to fulfill data processing needs across diverse scenarios. Leveraging anomaly detection, pattern recognition, and other sophisticated techniques, it precisely identifies aberrant behaviors and potential threats within the network, thereby providing robust support for the development of security early warning and defense strategies ^[9]. The design of network security data processing and analysis module takes full account of scalability and flexibility. An open architecture and a standardized interface enable the module to be integrated with other security devices and systems for data sharing and collaborative defense. Fig. 2 shows the network security data processing module.

Fig. 2. Network security data processing module.

In terms of security, the module takes strict security protection measures from data transmission, storage and processing. Module also regularly conducts security audit and vulnerability scanning, and finds and deals with security problems in time, to ensure stability and reliability of the system. The module is scalable and flexible to integrate with other security devices and systems to provide strong support for network security defense ^[10].

2.3 Network Security Situation Display and Early Warning Module Design

The network security situation displays and warning module is the core component of the network security situation awareness platform. It utilizes technologies such as data preprocessing, anomaly detection, and behavior analysis to achieve perception and understanding of network security conditions. By cleaning and formatting the collected network data, the accuracy and reliability of the data are ensured ^[11]. By using techniques such as data mining and machine learning, abnormal behavior and potential threats in the network can be identified, providing effective support for network security defense ^[12,13].

Regarding visual display, the warning module offers a vivid depiction of the current network security status through interactive charts, maps, or dashboards. This enables users to gain a clear comprehension of crucial information such as network traffic, attack sources, and types. Additionally, it provides comparative and trend analyses of historical data, further assisting users in comprehending the overall network security situation.

In order to achieve real-time monitoring and early warning, the early warning module has established corresponding mechanisms to continuously monitor and analyze network data. Once abnormal behavior or potential threats are detected, the early warning mechanism is immediately triggered. Notify relevant personnel or systems in a timely manner through email, SMS, or push notifications, in order to take timely response measures ^[14].

By managing knowledge and rule bases, known threat information, attack patterns, and defense measures are stored to assist in analysis and judgment. Users can customize and customize according to their needs, setting specific monitoring rules, alarm thresholds, and display methods to meet personalized network security situational awareness needs.

The network security situation displays and warning module conducts a comprehensive analysis and processing of network data, enabling the perception and alerting of network security situations. This module provides crucial support for network security defense, assisting users in promptly identifying and responding to network security threats, ultimately enhancing their network security defense capabilities. In design, by considering the intuitiveness of data visualization and personalized customization of user experience, the practicality and usability of the module are improved.

3.1 Application of Big Data Storage and Management Technology in Network Security

Traditional data processing methods face difficulties in handling large-scale, highly concurrent network data ^[15,16]. With the help of big data processing frameworks such as Hadoop and Spark, massive amounts of data can be efficiently processed and analyzed. These frameworks can uncover security information and attack patterns hidden in data through techniques such as correlation analysis, anomaly detection, and pattern recognition.

Real time processing capability is an indispensable part of network security. By monitoring network traffic and events in real-time, detecting abnormal behavior in a timely manner, and issuing warnings, network attacks can be effectively prevented ^[17]. By using stream processing technology, big data can achieve high-speed data collection, analysis, and early warning, greatly improving the response speed to security incidents ^[18,19].

This article designs large-scale storage management technologies to handle large-scale data storage and access. Simultaneously optimize performance to achieve optimal data read and write speeds. Fig. 3 shows the design of large-scale storage management technology, which includes key elements such as distributed storage systems, data sharding, and data replication. In terms of distributed storage systems, distributed file systems and object storage systems are used to achieve reliable storage and efficient access to data. By storing data shards on multiple nodes and using redundancy mechanisms to ensure data reliability and fault tolerance. Data sharding entails breaking down a massive dataset into numerous smaller segments to facilitate enhanced parallel processing and storage. Each shard can be handled independently on different nodes, significantly enhancing overall processing efficiency and concurrency. In parallel, data replication strives to boost the reliability and fault tolerance of the data. By replicating data to different nodes, data availability can be maintained even in the event of a node failure. Simultaneously improving read performance through data replication enables parallel access of data across multiple nodes.

Fig. 3. Big data storage and management technology design.

3.2 Application of Data Analysis and mining in network security

With the expansion of the network scale and the diversification of attack means, the traditional security defense means have been difficult to deal with ^[20]. Big data analysis technology, with its powerful data processing ability and deep mining ability, provides a new solution for network security defense.

By conducting comprehensive monitoring and analysis of network traffic, user behavior, and log data, big data technology is capable of promptly detecting abnormal behaviors, pinpointing potential threats, and offering robust support for the development of security early warning and defense strategies. First, anomaly detection is one of the important applications of big data analysis in network security ^[21]. By establishing a baseline model of normal behavior, big data analysis can monitor the abnormal fluctuations of network traffic and the abnormal changes of user behavior in real time, and find out the potential aggressive behavior or malicious traffic in time. This defense strategy based on anomaly detection can effectively reduce underreporting and false positives and improve the discovery rate of safety events. By analyzing IP addresses, port numbers, protocol and other information in network traffic, potential botnets and malware propagation threats can be associated. Big data analysis and mining technology can also trace and analyze network attacks to provide key evidence for event disposal ^[22]. By deeply excavating the traces left in the process of network attack, we can track the behavior path and the source of the attack of the attacker ^[23]. Data privacy and security issues should be addressed during the application process, and the performance and efficiency of the technology are constantly optimized ^[24]. The data sensitivity index (DS) formula is shown in (1).

3.3 Application of Artificial Intelligence Technology in Network Security Situational Awareness

AI technology can accurately identify abnormal behaviors, predict potential threats, and provide strong support for security decisions ^[25]. Artificial intelligence technology provides a powerful analytical capability for network security situational awareness. Traditional security analysis methods are often difficult to deal with the complex network environment and diversified attack means. By constructing the abnormal detection model, artificial intelligence can monitor the network traffic and user behavior in real time, detect the abnormal patterns in time, and effectively improve the discovery rate of security events ^[26]. AI technology enables threat hunting and malware analysis. Through a comprehensive analysis of network traffic and log data, AI technology can find hidden attack clues and further track the source of attacks. Through a combination of static and dynamic analysis, AI technology swiftly identifies malware families, functionalities, and transmission routes, thereby furnishing crucial evidence for the prompt resolution of security incidents. In addition, artificial intelligence technology is also widely used in the visual display of network security situation. By presenting complex and multi-dimensional network security situation information, artificial intelligence technology can provide intuitive and easy to understand display interface. This helps security personnel to quickly understand the overall security situation of the network and make targeted security decisions. AI technology possesses the capability to learn from historical data and security events, continually optimizing the content and layout displayed, ultimately enhancing user experience. However, AI technology also faces some challenges in the application of network security situational awareness. The quality and integrity of the data is essential for the accuracy of the AI analysis. Threat Identification Index (TR) and Safety Response Index (SR) formulas are shown in (2) and (3).

Challenges such as data quality and privacy protection should be focused on during the application process, and the performance and efficiency of the technology should be continuously optimized ^[27]. In practical application, artificial intelligence technology combines with big data, cloud computing and other advanced technologies to build a strong network security protection system. For example, a cloud-based security service platform can integrate multi-source data, provide intelligent analysis, timely warning, and disposal of security incidents. At the same time, combined with data encryption, identity authentication and other traditional security measures, to form a multi-level, all-round security protection system. The total threat score (TS) and network security index (CS) formulas are shown in (4) and (5).

4.1 Application Case Analysis of the Network Security Situational Awareness Platform in the Financial Industry

In practical application, the platform shows significant application value. First of all, in terms of abnormal detection, the platform monitors network traffic and user behavior in real time, and finds abnormal traffic patterns and potential malicious behaviors ^[28]. Through the automatic blocking or alarm function, the risk of network attack is effectively reduced. Secondly, in terms of threat hunting and traceability analysis, the platform comprehensively analyzes network traffic and log data, finds hidden attack clues, tracks the attack source, and provides detailed attack path and tool analysis for the security team, which is helpful to quickly locate and handle security incidents. Furthermore, the platform offers a user behavior analysis feature that tracks and scrutinizes users' network actions to promptly identify potential security hazards, such as data breaches or unauthorized access. These functionalities expedite the implementation of timely interventions and corrective measures. The platform also has the function of network security visualization. By visualizing the complex data, users are provided with an intuitive network security situational awareness interface. This not only helps to quickly understand the overall security situation of the network, but also provides strong data support for decision makers. For example, the safety dashboard can display cyber threat maps, traffic trends and other information in real time, providing strong support for the security team's rapid response ^[29]. By introducing the network security situation awareness platform, the bank achieves the following aspects: first, it improves the discovery rate and disposal efficiency of security incidents; second, it reduces the network security risk and reduces potential losses; third, it improves the user behavior analysis ability and strengthens the internal management and supervision; fourth, it provides the network security visualization interface to facilitate the work of decision makers and security teams. However, the platform also faces some challenges in practical application ^[30].

4.2 Experimental Analysis

The experimental steps are divided into data acquisition, data processing, data analysis, platform application, visual demonstration, and experimental evaluation. Fig. 4 shows the amount of data obtained through different forms.

Fig. 4. The amount of data obtained through the different forms.

Utilizing technologies such as network traffic mirroring and network data packet capture, data collection gathers network traffic data and log data. The amassed data encompasses network traffic, system logs, application logs, and more. Since the platform's inception in 2018, it has accumulated 650TB of original network security data. Remarkably, in 2023, the platform's average daily influx of new data reached 10GB, representing a fivefold increase compared to 2018. Data processing the collected raw data is cleaned, reweighted, classified and processed to eliminate erroneous data and duplicate data and provide high-quality data sources for subsequent analysis. Analysis content includes traffic mode recognition, abnormal detection, threat early warning and so on. The total perception score (AS) is shown in (6).

Visual display will show the analysis results in a visual way, such as the network topology map, security situation map, etc., so that users can intuitively understand the network security situation. The configuration and algorithm parameters of the platform are adjusted according to the actual demand to improve the accuracy of monitoring and the timeliness of early warning. Experimental evaluation evaluates the function and performance of the network security situational awareness platform according to the actual application effect. The evaluation indicators include the real-time performance, accuracy and stability of the platform. Fig. 5 Statistics of different cyber-threat data.

Fig. 5. Statistics of different network threat data.

Through experimental verification, the network security situational awareness platform based on big data can effectively monitor the network traffic and log data, and find the potential security threats and attack behaviors. The following is an analysis of the experimental results:

Traffic statistics: During the 30 days, the average daily traffic of the enterprise network was 160GB, and the highest daily traffic reached 220 GB. Flow peaks mainly on weekdays from 9-11 am and 2-4 PM. The usability index (U) formula is shown in (7).

The resource utilization index (R) is shown in formula (8).

Threat Identification: Following rigorous data analysis and mining, we have pinpointed 2,300 potential security threat events. Of these, malware attacks constitute 45%, DDoS attacks account for 30%, phishing sites make up 15%, and the remaining 10% comprises other threats.Total efficacy score (E) is shown in formula (9).

Response time: The average response time of the platform is 3.5 seconds, and the maximum response time does not exceed 10 seconds. This indicates a good real-time performance and responsiveness of the platform. Fig. 6 Comparison of the response and disposal time when facing different networks.

Fig. 6. Comparison of response and disposal time for different networks.

In terms of data analysis, the platform uses big data analysis technology to conduct in-depth analysis of the processed data. Through traffic mode identification, abnormal detection and threat warning, potential security threats and attacks can be found in time. At the same time, through machine learning and data mining and other technologies, can automatically learn and optimize security strategies, improve the accuracy and timeliness of early warning. As shown in the formula (10).

In terms of visual display, the platform presents the analysis results to the users in a visual way. Through the intuitive network topology map and security situation map and other display means, users can quickly understand the network security situation and make the corresponding security decisions. This improves the efficiency and effect of safety management. User feedback score (FS) and total integration score (IS) formula is shown in (11).

The total integration score (IS) is shown in (12).

Regarding platform application, the developed network security situational awareness platform will be implemented in real-world environments for continuous security monitoring. During practical deployment, the platform's configuration and algorithm parameters will be fine-tuned in accordance with the evolving needs of users and network environments, thus enhancing the precision of monitoring and the promptness of early warnings. At the same time, the platform has strong scalability and customization for different security scenarios and requirements. Fig. 7 shows the resource allocation and utilization diagram.

In terms of experimental evaluation, the function and performance of the network security situational awareness platform are evaluated according to the actual application effect. The evaluation results show that the platform has strong real-time performance, accuracy and stability. In the face of a complex network environment and diversified security threats, the platform can provide comprehensive and accurate monitoring and early warning services.

Fig. 7. Resource allocation and utilization diagram.