1. Operation Description for Distance Calculation

Distance calculation is a fundamental operation in vector similarity search, used to quantify the similarity or dissimilarity between vectors in high-dimensional spaces. This work focuses on three commonly used distance metrics: Manhattan distance, Euclidean distance, and cosine similarity.

Manhattan distance, also known as the L1 distance, measures the sum of the absolute differences between the corresponding vector elements p and q. It is defined as

Euclidean distance, also referred to as L2 distance, represents the straight-line distance between two points p and q in Euclidean space. It is calculated as

Cosine similarity measures the cosine of the angle θ between two vectors p and q, indicating their directional alignment regardless of magnitude. It is computed as

where p and q are vectors, $p \cdot q$ is the dot product, and $\|p\|$, $\|q\|$ are their norms. A value closer to 1 implies a higher similarity.

Euclidean distance measures the separation between two vectors, but vector search typically omits the square root since only relative distances matter for ranking.

Since most of the datasets of vector searches using cosine similarity are already normalized to the size of the two vectors as 1, we implemented the cosine library the same as dot product.

2. Related PIM Architectures

Recent years have witnessed the emergence of diverse DRAM-based Processing-in-Memory (PIM) architectures. SK hynix AiM extends the HBM/GDDR interface with all-bank parallelism and in-memory GEMV support, targeting matrix-vector multiplication workloads ^[10,11]. UPMEM’s DIMM-PIM integrates lightweight processing units (DPUs) inside DDR4 DIMMs, offering flexible near-memory acceleration for a wide range of tasks ^[12-14]. While these efforts demonstrate the broad potential of DRAM-PIM, our focus differs: distance computation in vector similarity search requires repeated execution of simple arithmetic (ADD, MUL). For this purpose, Samsung’s HBM-PIM, which natively exposes such primitive operations, provides a more direct and efficient baseline. We therefore adopt HBM-PIM to align with architectures recently offered by leading memory vendors while ensuring suitability for distance-calculation workloads ^[15,16]. Based on this rationale, the next subsection describes the baseline Samsung HBM-PIM architecture that serves as the foundation of our work.

3. Baseline HBM-PIM Architecture

Fig. 2 overviews baseline Samsung PIM architecture ^[15,16]. The architecture consists of Even and Odd banks that share a single PIM unit, so they cannot perform operations simultaneously. The PIM execution unit is composed of a Floating-Point Unit (FPU), Command Register Files (CRF), General Register Files (GRF), and Scalar Register Files (SRF). The architecture supports three modes: single bank mode, all bank mode, and all bank-PIM mode. In single bank mode, it operates in standard DRAM mode, performing conventional DRAM commands. In all-bank mode, it can access multiple banks simultaneously, while in all-bank PIM mode, it triggers PIM commands and executes PIM operations. Currently, Samsung PIM architecture supports nine instructions: four ALU instructions (ADD, MUL, MAC, MAD), two Data instructions (MOV, FILL), and three Control instructions (NOP, JUMP, EXIT).

Fig. 2. Baseline PIM architecture [15,16].

As shown in Fig. 3, the programmable computing unit, referred to as the PIM unit, consists of a FPU, CRF, GRF, and SRF. The FPU can execute 16 operations on 16-bit data simultaneously. PIM instructions are stored in the CRF, so PIM commands are triggered as specified by the CRF. The GRF is divided into GRF_A and GRF_B, which are connected to the Even and Odd banks, respectively. Each GRF consists of eight 16-bit registers, allowing storage of eight 16-bit elements. Lastly, the SRF is divided into SRF_A, for addition operations, and SRF_M, for multiplication, with each section containing eight 16-bit registers, similar to the GRF.

Fig. 3. The micro-architecture of programmable computing unit.

This work adopts the Samsung HBM-PIM architecture as the baseline to ensure alignment with prior work and provide a fair reference point for performance evaluation. Consequently, 16-bit data types are used throughout the system. Although many public datasets are originally provided in 32-bit floating point (FP32) format, we confirm that quantization to 16-bit precision preserves accuracy in our target workloads ^[17]. Specifically, the SIFT dataset consists of image descriptors with integer values ranging from 0 to 255, which are naturally robust to reduced precision and show no degradation even under BF16 representation. For the GIST dataset, we empirically evaluated vector search accuracy after BF16 quantization and observed no measurable drop compared to FP32. Therefore, implementing the PIM unit with FP16 capability efficiently supports both neural network operations and vector similarity search, while remaining practical for real-world datasets.

1. Software Implementation

The most used distance metrics for calculating vector similarity are Euclidean distance, cosine similarity, and dot product. However, in datasets designed for cosine similarity-based vector search, the vectors are often normalized. Therefore, in this study, we considered cosine similarity and dot product as one metric and developed libraries for PIM devices to compute Euclidean distance and cosine similarity ^[18-20]. Since the goal is to compare relative distances rather than absolute ones, square root operations can be omitted in Euclidean distance calculation.

Vector similarity search compares the query vector with database vectors and finds nearest neighbor vectors. In this study, we implemented distance operation libraries for the vector similarity search in PIM environments. First, the database vectors and query vectors are written to memory regions referred to as srcA and srcB, which are then mapped to the PIM address space. The PIM unit performs distance calculations between these regions, and the results are stored in dstC. Finally, the host uses dstC to classify nearest neighbors. Since these libraries access datasets from PIM space and compute distances in its unit, they save data transfer latency between memory and host and do not require additional memory space. Fig. 4 shows the resulting distance computation library with Euclidean distance and cosine similarity, used for vector similarity search.

Fig. 4. Distance computation library for Euclidean distance and cosine similarity.

2. System Architecture

We initially implemented the distance calculation at the software level. However, experimental results revealed performance bottlenecks due to the limitations of the existing PIM instruction set. Therefore, we performed hardware-level optimization by modifying the datapath and extending the instruction set ^[21]. Accordingly, we implemented the Euclidean distance computation, and through the datapath modification, we found that it was also possible to optimize the Manhattan distance computation. As a result, we implemented two instructions to support both Euclidean and Manhattan distance computations. Since cosine similarity is already efficiently supported by the existing MAC instruction, no additional optimization was necessary. The proposed Instruction Set Architecture is shown in Table 1.

As shown in Fig. 5, we customized the instruction by adding a 1-bit 'm' field to determine whether to take the absolute value. Architecturally, an 'absolute value unit' is needed in the FPU_ADD unit. The absolute value unit functions by changing the most significant bit (MSB) to 0. When the 'MAN' operation is triggered, the subtraction between the two vectors is calculated, the absolute value is taken, and the result is accumulated in the GRF.

As shown in Eq. (2), the L2 operation calculates the subtraction between the two vectors, squares that value, and then accumulates the result. However, since the result needed to be stored in the GRF after the ADD, only 8 elements could be stored at a time, requiring frequent data transfers to the banks. To address this, an operation that performs the multiplication immediately after the addition, without the need to store the intermediate result, was proposed. This led to the creation of a new instruction, "AMC" (Add-Multiply-Accumulate), like MAC (Multiply-Accumulate), but with the order of operations reversed (ADD to MUL). To implement this, a new path connecting the FPU_ADD to the FPU_MUL is required. By implementing this, the operation could be performed without transferring data between the banks.

Fig. 5. Instruction format to support multiple similarity metrics.

1. Application-level Evaluation on DRAM-based PIM System

The initial evaluation was conducted on a VMK180 board, configured with LPDDR4 as system memory and two GDDR6 modules as PIM memory ^[22]. A dual-core ARM Cortex-A53 processor from Zynq was used, with only a single core utilized during the experiments. The vector similarity search application was implemented using a brute-force algorithm under the K-Nearest Neighbor (KNN) search category, and the proposed distance computation library was evaluated within this framework. The experiments employed random datasets of various sizes. Figs. 6 and 7 present the results of brute-force search using Euclidean distance and cosine similarity, respectively, on both CPU and PIM environments. In the brute-force search, distances between the query vector and all vectors in the database were computed, and the top 100 closest neighbors were returned after sorting.

The results showed that the PIM outperformed the CPU in both Euclidean distance and cosine similarity computation. Specifically, for Euclidean distance, the PIM achieved an average speed improvement of 44.2% over the CPU, while for cosine similarity, the improvement was 59.0%. The performance gap between PIM and CPU increased as the dataset size grew, with cosine similarity exhibiting a relatively higher improvement rate compared to Euclidean distance.

Fig. 6. Computing time with log scale for brute-force with Euclidean distance.

Fig. 7. Computing time with log scale for brute-force with cosine similarity.

2. Instruction-level Optimization on HBM-based PIM Simulator

Although application-level implementation and evaluation were performed on the DRAM-based PIM system, it exposed inherent limitations ^[23] of the existing instruction set, which was not optimized for vector distance computations. While it was technically feasible to implement distance computations using the existing instructions, frequent data movement and intermediate storage across memory banks resulted in performance bottlenecks, preventing full utilization of PIM’s inherent performance benefits. To address these challenges, we transitioned the evaluation platform to an HBM-based PIM simulator environment ^[24,25], which provided greater flexibility for instruction set design and datapath modification. The HBM-based PIM simulator was configured by integrating DRAMSim2, and the performance of the proposed AMC and MAN instructions (optimized for Euclidean distance and Manhattan distance computations, respectively) was evaluated. The simulator architecture is illustrated in Fig. 8. The host processor generated transactions required for PIM operations, which were executed by the PIM unit within the simulator. DRAMSim2 measured the number of cycles required for each transaction, and upon completion of the simulation, both the computation results and cycle counts were returned to the host.

Fig. 8. The system organized with PIM simulator with DRAMSim2.

As shown in Table 2, the total cycle count for Euclidean distance computation using the AMC instruction was reduced by up to 44% compared to using the existing instruction. Additionally, Fig. 9 shows that the performance improvement increased as the input matrix size grew. Notably, for matrix sizes of 4096 × 4096 and above, the performance of the Manhattan distance computation using the MAN instruction in the PIM-enabled state surpassed that of the PIM-disabled state, as shown in Table 3.

Fig. 9. Performance gain when using the baseline instruction and AMC.

This progression from application-level evaluation on a DRAM-based platform to instruction-level optimization on an HBM-based platform demonstrates that architectural improvements are necessary to fully exploit the potential of Processing-In-Memory for workloads such as distance computations.

To assess the implementability of the proposed PIM datapath, we conducted an area comparison using a commercial digital library. Both the baseline HBM-PIM datapath and our proposed datapath were implemented at the RTL level and synthesized using the TSMC 65nm standard cell library. The synthesis results show that the proposed design incurs less than 5% area overhead compared to the baseline. This minimal overhead suggests that the proposed datapath is practically implementable without significant hardware cost.