Mobile QR Code QR CODE : Journal of the Korean Society of Civil Engineers

  1. 성균관대학교 글로벌스마트시티융합공학과 석박사 통합과정 (Sungkyunkwan University․papawinaung1995@gmail.com)
  2. 종신회원․성균관대학교 미래도시융합공학과 연구교수, 공학박사 (Sungkyunkwan University․ycleedh@gmail.com)
  3. 성균관대학교 건설환경시스템공학과 박사과정, 공학석사 (Sungkyunkwan University․mitjy26@gmail.com)
  4. 정회원․성균관대학교 미래도시융합공학과 석사과정 (Sungkyunkwan University․raonik6713@naver.com)
  5. 종신회원․교신저자․성균관대학교 건설환경공학부 교수, 공학박사 (Corresponding Author․Sungkyunkwan University․shparkpc@skku.edu)



메쉬 경량화, k-최근접이웃 (KNN), 빌딩 정보 모델링 (BIM), 증강현실 (AR)
Mesh optimization, K-nearest neighbors (KNN), Building information modeling (BIM), Augmented reality (AR)

1. Introduction

Currently, construction industry projects are becoming more complex and more difficult to control (Machado and Vilela, 2020). As construction projects become larger and more sophisticated, various technologies are being developed to manage the vast amount of information generated by these projects more efficiently. Among the various technologies in use at present, augmented reality (AR) and building information modeling (BIM) are expanding in the construction industry. According to a survey by the UK’s fastest-growing Building Information Modeling (BIM) library, the BIM utilization rate was only 13 % in 2011, though it increased to 73 % in 2020 (NBS Enterprises Ltd., 2020). And as a result of examining the augmented reality technology, widely used in many fields currently, the market size is expected to grow by 2025, and this technology is expected to be widely used in the construction and industrial fields (Software Policy and Research Institute, SPRI).

AR provides virtual information on physical surfaces, assisting with interactions with the real environment through a combination of virtual information and real information (Karji et al., 2017). BIM technology utilizes parametric modeling technology to link and manage various 3D shape information and attribute information of a construction project, not only providing functions such as automatic drawing creation, quality improvement, quantity calculation, process management, and inter-process interference reviews but also making various simulations possible in the design and construction stages (An et al., 2010). Integration between BIM and AR can serve as a reliable tool for visualizing the construction and improving the identification, handling and communication of progress discrepancies at the construction site. Numerous means by which to integrate AR with BIM have been proposed by the scholars. For example, Chai et al.(2019) showed that BIM is compatible and thus can be integrated with the AR platform through their proposed AR-BIM system.

Various methods already exist for projecting BIM data into AR (An et al., 2010). In order to convert a BIM model to AR, the created BIM model must be transferred from each BIM authoring tool (e.g., Revit, Civil 3D) to one game engine (e.g., Unity, Unreal). In addition, conversion to a neutral format such as FBX, OBJ, and IFC is essential to create a single unified model based on it. However, during the process of converting the BIM model to a neutral format, the shape of the model can break, become deformed, or is duplicated frequently, which can seriously degrade the performance when running in an AR environment. To solve this problem, it is necessary to reduce the number of meshes in the BIM model by using commercial software or by developing a program directly.

The most dominant and fastest approach to keeping the overall polygon count low is to use a mesh optimization algorithm to generate a low-poly version of an existing high-poly model (Bahirat et al., 2018). From 1993 to the present (2021), many researchers have conducted mesh optimization studies (Hoppe et al., 1993). However, most existing mesh simplification algorithms are complex and are limited in their ability to handle meshes properly while maintaining the baseline shape of the 3D model (Luebke, 2001). Therefore, this study developed an algorithm to reduce the mesh of BIM models by utilizing deep learning for high- performance AR visualization (see Fig. 1).

Fig. 1. Conceptual Framework for the Mesh Optimization Algorithm
../../Resources/KSCE/Ksce.2022.42.2.0249/fig1.png

2. Technical Background of Relevant Methods for Mesh Optimization

2.1 K-nearest Neighbors

K-nearest neighbors (KNN) is a supervised and pattern classification learning algorithm that helps us find the class of the new input (test value) when k-nearest neighbors are chosen, with the distance then calculated between them. KNN is useful for classification and regression owing to its versatility (Cao et al., 2019). The k-nearest-neighbor classifier is usually based on the Euclidean distance between a test sample and the specified training samples (Peterson, 2009). Let $x_{i}$ be an input sample with p features $(x_{i1},\: x_{i2},\: ...,\: x_{ip})$, n be the total number of input samples (i=1,2,…,n) and p be the total number of features (j=1,2,…,p). The Euclidean distance between sample $x_{i}$ and $x_{l}$ (l=1,2,…,n) is then defined as shown below.

(1)
$d(x_{i},\: x_{l})=\sqrt{(}x_{i 1}- x_{l1})^{2}+(x_{i 2}- x_{l2})^{2}+ ... +(x_{i p}- x_{lp})^{2}$

A graphic depiction of the nearest neighbor concept is illustrated by the Voronoi tessellation (Peterson, 2009) depiction shown in Fig. 2.

Fig. 2. Voronoi Tessellation Showing Voronoi Cells of 19 Samples Marked with a "+"(Peterson, 2009)
../../Resources/KSCE/Ksce.2022.42.2.0249/fig2.png

2.2 Building Information Modeling

Building Information Modeling (BIM) is a digital model that expresses the shape and properties of a facility as information based on a three-dimensional model to integrate and utilize all information that occurs throughout the life cycle of the facility. The British Standards Institute defines BIM as the process of generating and managing information about a building during its entire life to form a ‘system’, at the heart of which is a component- based 3D representation of each building element (Winfield, 2015). BIM was introduced over a decade ago mainly to differentiate the information-rich architectural 3D modeling from traditional 2D drawings, and it has been acclaimed by its advocates as a lifesaver for complicated projects owing to its ability to correct errors early in the design stage and accurately schedule construction (Kubba, 2012).

Recently, various BIM technology-based simulation tools have been developed for efficient communication between the planning, design, construction, and maintenance phases of a construction project and to ensure consistent project performance, but BIM has made only a limited contribution to research in this field due to the limited interaction between the real world and the virtual world (Chernick et al., 2020). However, integrating AR into BIM systems provides a platform on which to achieve interactivity and can be helpful with regard to decision-making and efficiency of processing due to its capacity for real-time visualization (Wang et al., 2014).

2.3 Augmented Reality

Augmented Reality (AR) refers to technologies and experiences that bring computer-generated objects into the user’s physical (real-time) environment (Song et al., 2021). AR is a technology that enhances work efficiency by augmenting virtual information in real time and allowing users to interact with augmented virtual information. Research on AR began in the 1960s when van Surtherland developed the first see-through Head Mounted Display (HMD), and in the early 1990s, Boeing coined the term ‘Augmented Reality’ (Korea Electronics and Telecommunications Research Institute). Generally, AR has become increasingly relevant as a research field and the technology is expected to reach a market size approaching 200 billion U.S. dollars by 2025 (Tatasciore, 2018).

Augmented reality uses a virtual environment created with computer graphics, but the main character is the real environment. Computer graphics play a role in providing the additional information necessary for the real environment. This means that the real environment and the virtual screen are blurred by overlapping the 3D virtual image on the actual image the user is viewing (Kim et al., 2013).

Behzadan and Kamat (2007) conducted a study to simulate a 3D object based on an image of a construction site, and it is expected that the user’s visual recognition environment when accepting real spatial information can be improved by using augmented reality. Currently, simultaneous and multifaceted collaboration is required as construction projects become larger and more complex from construction planning and design stages to maintenance (Jang et al., 2019). Therefore, it is necessary to check virtual buildings through AR technology and to induce decision-making and high participation of building users through design drawing revisions to achieve high performance.

3. Mesh Optimization Algorithm Approach

The mesh optimization algorithm was designed to create a low-poly version of a 3D model effectively and accurately, proposing triangle combining and vertex merging using reconstructions of common vertices for merged triangles. This type of algorithm can merge three triangles into one triangle using the triangle centroid principle via calculations that take the average of the x-coordinate, y-coordinate, and z-coordinate of the triangle vertex (see Fig. 3).

Fig. 3. Steps of the Optimization Process and Concept
../../Resources/KSCE/Ksce.2022.42.2.0249/fig3.png

Our algorithm is based on the Unity C\# code based on the mesh finder and mesh level of details (LOD) technique, which is capable of finding, merging and reconstructing the triangular vertices of the 3D model using the triangular centroid calculation principle composed of three points. We used the triangle center principle in the mesh finder code, which counts the vertices and triangles of the model and draws a line between the vertices and reconstructs three center points. In the mesh LOD code, codes were written to add the ability to export the mesh code and divide the model into whole/part models.

Unity, a cross-platform game engine developed by Unity Technologies, can be used to create three-dimensional (3D) and two-dimensional (2D) games as well as interactive simulations and other experiences and can be used for video applications, such as those related to automobiles, architecture, engineering and construction (Xie, 2012). It has been adopted in industries other than gaming. To call it from Unity, you need to attach a script to a Game Object (3D model) in your scene.

Scripts are written in a special language understandable by Unity, and the language used by Unity is called C\#. The written code C\# in our algorithm is responsible for finding, merging, and reconstructing the triangle vertices in the 3D model. For the final step, we used a KNN to test the classification based on the 3D model vertex distance dataset. A flow diagram of the proposed algorithm is shown in Fig. 4.

Fig. 4. Flowchart of the Proposed Algorithm
../../Resources/KSCE/Ksce.2022.42.2.0249/fig4.png

4. Experiment on the proposed mesh optimization algorithm

4.1 Specifications of the Test Model and 3D Modeling

A steel glass structure bridge with a length of 34 m, a height of 5 m, and a width of 2 m connecting Sungkyunkwan University’s No.1 Engineering Building 23 and No. 2 Engineering Building 25 was selected as the target model for our algorithm. It was modeled in Revit and passed to the Unity engine using the Unity Reflect process. Unity Reflect for Autodesk Revit, industry-leading BIM software, enables designers and engineers to transfer Revit models into real-time 3D experiences (2021 Unity Technologies). It is possible to check all types of structure details as well as multiple meshes and components (see Fig. 5).

Fig. 5. Target Model 3D Modeling
../../Resources/KSCE/Ksce.2022.42.2.0249/fig5.png

4.2 Data Collection and Processing Dataset using KNN Classification

KNN algorithms are used to find k-nearest neighbors of data points in a dataset, and the basic method of finding k-nearest neighbors of a point is to compute all Euclidean distances using data points (Xia et al., 2015). The dataset is collected by running the target 3D model multiple times with a similar level of detail (Wang et al., 2021). We can determine the class to which a new input (test value) belongs when k-nearest neighbors are chosen, and the distance is calculated between them. In our model, there are thousands of combinations possible when merging vertices. Thus, we use the KNN to determine the best fit to define the distance between the vertices. As a result of extracting the distance dataset of points from the selection model using the unity C\# code, we received a data set with 590,942 rows and four columns.

For data processing using KNN classification, the collected data mentioned earlier are divided into a training dataset (80 %) and a test dataset (20 %) through the Python package Scikit-learn. After the dataset train-test-split process, we used the sklearn KNeighbors Classifier and KNN-predict to make predictions. Based on our collected dataset, we obtained 90 % accuracy (see Fig. 6).

Fig. 6. Processing Dataset Using KNN Classification
../../Resources/KSCE/Ksce.2022.42.2.0249/fig6.png

4.3 Results and Discussion

As a result of our algorithm testing the SKKU steel bridge 3D model, although there may be some difficulties in checking with the naked eye, as shown in Fig. 7, blue is the model before reduction and green is the model after reduction. Moreover, there appears to be no problem in utilizing lightweight and original models for AR visualization, as there are no significant differences visually.

Fig. 7. Result of BIM Model Mesh Optimization
../../Resources/KSCE/Ksce.2022.42.2.0249/fig7.png

In addition to the entire model before and after optimization, the number of points and triangles before and after optimization for each structure could be checked immediately. As shown in Table 1, the vertex of the original model can be reduced by approximately 56 % and the triangle by about 42 %.

Table 1. Optimized Percentage Result of the Vertices and Triangles of the Model

$\dfrac{model\;Type}{Structure\;Type}$

Original model

Optimized model

Optimized %

$$ \left(V=\frac{V_{2}}{V_{1}} \times 100 \%, T=\frac{T_{2}}{T_{1}} \times 100 \%\right) $$

Vertices 1

Triangles 1

Vertices 2

Triangles 2

Vertices

Triangles

Glass Structure

7214

4804

4405

1921

61 %

40 %

Tube Structure

20310

10438

10628

4174

52 %

40 %

Full model

590942

335564

330116

140167

56 %

42 %

5. Conclusion

Recently, with the introduction of augmented reality in the fourth industrial revolution era, the construction industry is developing along with information and communication technology. AR applied to the construction field is used during the process of reviewing 3D models for architectural designs and is also used for construction management at construction sites.

In this paper, we proposed a new mesh reduction method based on central-triangular vertex decimation combined with a smart mesh finder and mesh LOD in order to automate the process of vertex merging by utilizing a KNN classification algorithm to optimize the imported BIM model with meta-data from an external program to visualize AR effectively. The classification operation well works on multi-hierarchy component 3D models and meshes, allowing it easily to find the most necessary parts for optimization. Some difficulty exists with regard to creating an easier import- export pipeline due to various file formats and corresponding underlying systems. This newer approach improves the topology preservation, and thus it is the best possible mesh reduction and reconstruction strategy. This study aimed to optimize the imported BIM model with meta data from an external program to visualize AR effectively, even on mobile devices with resource constraints.

For future work, the trained model will be uploaded onto a dedicated server for greater accessibility and availability. More diverse and complex mesh model testing will also be conducted for more accuracy. In addition, the optimized BIM models will be tested with an AR visualization device.

Acknowledgement

This research was conducted with the support of the “National R&D Project for Smart Construction Technology (No.21SMIP- A158708-02)” funded by the Korea Agency for Infrastructure Technology Advancement under the Ministry of Land, Infrastructure and Transport, and managed by the Korea Expressway Corporation and supported by 「Innovative Talent Education Program for Smart City」.

This manuscript is a revised version of the 2021 CONVENTION paper.

References

1 
An J. Y., Choi J. M., Kwon S. H., Song D. H., Ock J. H. (2010). "A study on the development of the project management system based on augmented reality.", Journal of the Korea Academia-Industrial Cooperation Society, Vol. 11, No. 8, pp. 3083-3093 (in Korean)DOI
2 
Bahirat K., Lai C., Mcmahan R. P., Prabhakaran B. (2018). "Designing and evaluating a mesh simplification algorithm for virtual reality.", ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Vol. 14, No. 3s, pp. 1-26DOI
3 
Behzadan A. H., Kamat V. R. (2007). "Georeferenced registration of construction graphics in mobile outdoor augmented reality.", Journal of Computing in Civil Engineering, Vol. 21, No. 4, pp. 247-258DOI
4 
Cao M. W., Li L., Xie W. J., Jia W., Lv Z. H., Zheng L. P., Liu X. P. (2019). "Parallel K nearest neighbor matching for 3D reconstruction.", IEEE Access, Vol. 7, pp. 55248-55260DOI
5 
Chai C., Mustafa K., Kuppusamy S., Yusof A., Lim C. S., Wai S.H. (2019). "Bim integration in augmented reality model.", International Journal of Technology, Vol. 10, No. 7, pp. 1266-1275DOI
6 
Chernick A., Morse C., London S., Li T., Ménard D., Cerone J., Pasquarelli G. (2020). "On-site BIM-enabled augmented reality for construction.", In The International Conference on Computational Design and Robotic Fabrication, Springer, Singapore, pp. 46-56DOI
7 
Hoppe H., DeRose T., Duchamp T., McDonald J., Stuetzle W. (1993). "Mesh optimization.", In Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques, Anaheim, California, USA, pp. 19-26DOI
8 
Jang M. H., Choi H. B., Nam K. W. (2019). "Method to use drawings and 3D models in building construction by augmented reality.", Journal of the Regional Association of Architectural Institute of Korea, ISSN, Vol. 21, No. 6, pp. 1229-5752 (in Korean)Google Search
9 
Karji A., Woldesenbet A., Rokooei S. (2017). "Integration of augmented reality, building information modeling, and image processing in construction management: a content analysis.", In AEI, pp. 983-992DOI
10 
Kim Y. T., Lee J. Y., Lee S. H., Choi J. S. (2013). "Construction and inspection management system using mobile augmented reality.", In The 19th Korea-Japan Joint Workshop on Frontiers of Computer Vision, IEEE, pp. 93-96DOI
11 
Kubba S. (2012). "Building information modeling, Handbook of Green Building Design and Construction", pp. 201-206DOI
12 
Luebke D. P. (2001). "A developer’s survey of polygonal simplification algorithms.", IEEE Computer Graphics and Applications, United States, Vol. 21, No. 3, pp. 24-35DOI
13 
Machado R. L., Vilela C. (2020). "Conceptual framework for integrating BIM and augmented reality in construction management.", Journal of Civil Engineering and Management, Vol. 26, No. 1, pp. 83-94DOI
14 
NBS Enterprises Ltd. (2020). "10th annual BIM report 2020", NBS, The Old Post Office, St Nicholas Street, Newcastle upon Tyne, NE1 1RH, UK.Google Search
15 
Peterson L. E. (2009). "K-nearest neighbor.", Scholarpedia, Vol. 4, No. 2, pp. 1883Google Search
16 
Song Y., Koeck R., Luo S. (2021). "Review and analysis of augmented reality (AR) literature for digital fabrication in architecture.", Automation in Construction, Vol. 128, pp. 103762DOI
17 
Tatasciore D. (2018). "DelivAR: An augmented reality mobile application to expedite the package identification process for last-mile deliveries", Doctoral Dissertation, Thesis for: M. Sc. Digital Media, University of Bremen, Available Online.Google Search
18 
Wang B., Wu G., Zhao Q., Li Y., Gao Y., She J. (2021). "A topology-preserving simplification method for 3D building models.", ISPRS International Journal of Geo-Information, Vol. 10, No. 6, pp. 422DOI
19 
Wang J., Wang X., Shou W., Xu B. (2014). "Integrating BIM and augmented reality for interactive architectural visualisation.", Construction Innovation, Vol. 14, No. 4, pp. 453-476DOI
20 
Winfield M. (2015). "Building information modelling: The legal frontier-overcoming legal and contractual obstacles.", Society of Construction Law, pp. 18Google Search
21 
Xia S., Xiong Z., Luo Y., Dong L., Zhang G. (2015). "Location difference of multiple distances based k-nearest neighbors algorithm.", Knowledge-Based Systems, Vol. 90, pp. 99-110DOI
22 
Xie J. (2012). "Research on key technologies base Unity 3D game engine.", International Conference on Computer Science and Education (ICCSE), IEEE Access, pp. 695-699DOI