빠 빠 윈 아웅
(Pa Pa Win Aung)
1
이동환
(Donghwan Lee)
2
박주영
(Jooyoung Park)
3
조민건
(Mingeon Cho)
4
박승희
(Seunghee Park)
5†
-
성균관대학교 글로벌스마트시티융합공학과 석박사 통합과정
(Sungkyunkwan University․papawinaung1995@gmail.com)
-
종신회원․성균관대학교 미래도시융합공학과 연구교수, 공학박사
(Sungkyunkwan University․ycleedh@gmail.com)
-
성균관대학교 건설환경시스템공학과 박사과정, 공학석사
(Sungkyunkwan University․mitjy26@gmail.com)
-
정회원․성균관대학교 미래도시융합공학과 석사과정
(Sungkyunkwan University․raonik6713@naver.com)
-
종신회원․교신저자․성균관대학교 건설환경공학부 교수, 공학박사
(Corresponding Author․Sungkyunkwan University․shparkpc@skku.edu)
Copyright © 2021 by the Korean Society of Civil Engineers
키워드
메쉬 경량화, k-최근접이웃 (KNN), 빌딩 정보 모델링 (BIM), 증강현실 (AR)
Key words
Mesh optimization, K-nearest neighbors (KNN), Building information modeling (BIM), Augmented reality (AR)
1. Introduction
Currently, construction industry projects are becoming more complex and more difficult
to control (Machado and Vilela, 2020). As construction projects become larger and more sophisticated, various technologies
are being developed to manage the vast amount of information generated by these projects
more efficiently. Among the various technologies in use at present, augmented reality
(AR) and building information modeling (BIM) are expanding in the construction industry.
According to a survey by the UK’s fastest-growing Building Information Modeling (BIM)
library, the BIM utilization rate was only 13 % in 2011, though it increased to 73
% in 2020 (NBS Enterprises Ltd., 2020). And as a result of examining the augmented reality technology, widely used in many
fields currently, the market size is expected to grow by 2025, and this technology
is expected to be widely used in the construction and industrial fields (Software
Policy and Research Institute, SPRI).
AR provides virtual information on physical surfaces, assisting with interactions
with the real environment through a combination of virtual information and real information
(Karji et al., 2017). BIM technology utilizes parametric modeling technology to link and manage various
3D shape information and attribute information of a construction project, not only
providing functions such as automatic drawing creation, quality improvement, quantity
calculation, process management, and inter-process interference reviews but also making
various simulations possible in the design and construction stages (An et al., 2010). Integration between BIM and AR can serve as a reliable tool for visualizing the
construction and improving the identification, handling and communication of progress
discrepancies at the construction site. Numerous means by which to integrate AR with
BIM have been proposed by the scholars. For example, Chai et al.(2019) showed that BIM is compatible and thus can be integrated with the AR platform through
their proposed AR-BIM system.
Various methods already exist for projecting BIM data into AR (An et al., 2010). In order to convert a BIM model to AR, the created BIM model must be transferred
from each BIM authoring tool (e.g., Revit, Civil 3D) to one game engine (e.g., Unity,
Unreal). In addition, conversion to a neutral format such as FBX, OBJ, and IFC is
essential to create a single unified model based on it. However, during the process
of converting the BIM model to a neutral format, the shape of the model can break,
become deformed, or is duplicated frequently, which can seriously degrade the performance
when running in an AR environment. To solve this problem, it is necessary to reduce
the number of meshes in the BIM model by using commercial software or by developing
a program directly.
The most dominant and fastest approach to keeping the overall polygon count low is
to use a mesh optimization algorithm to generate a low-poly version of an existing
high-poly model (Bahirat et al., 2018). From 1993 to the present (2021), many researchers have conducted mesh optimization
studies (Hoppe et al., 1993). However, most existing mesh simplification algorithms are complex and are limited
in their ability to handle meshes properly while maintaining the baseline shape of
the 3D model (Luebke, 2001). Therefore, this study developed an algorithm to reduce the mesh of BIM models by
utilizing deep learning for high- performance AR visualization (see Fig. 1).
Fig. 1. Conceptual Framework for the Mesh Optimization Algorithm
2. Technical Background of Relevant Methods for Mesh Optimization
2.1 K-nearest Neighbors
K-nearest neighbors (KNN) is a supervised and pattern classification learning algorithm
that helps us find the class of the new input (test value) when k-nearest neighbors
are chosen, with the distance then calculated between them. KNN is useful for classification
and regression owing to its versatility (Cao et al., 2019). The k-nearest-neighbor classifier is usually based on the Euclidean distance between
a test sample and the specified training samples (Peterson, 2009). Let $x_{i}$ be an input sample with p features $(x_{i1},\: x_{i2},\: ...,\: x_{ip})$,
n be the total number of input samples (i=1,2,…,n) and p be the total number of features
(j=1,2,…,p). The Euclidean distance between sample $x_{i}$ and $x_{l}$ (l=1,2,…,n)
is then defined as shown below.
A graphic depiction of the nearest neighbor concept is illustrated by the Voronoi
tessellation (Peterson, 2009) depiction shown in Fig. 2.
Fig. 2. Voronoi Tessellation Showing Voronoi Cells of 19 Samples Marked with a "+"(Peterson, 2009)
2.2 Building Information Modeling
Building Information Modeling (BIM) is a digital model that expresses the shape and
properties of a facility as information based on a three-dimensional model to integrate
and utilize all information that occurs throughout the life cycle of the facility.
The British Standards Institute defines BIM as the process of generating and managing
information about a building during its entire life to form a ‘system’, at the heart
of which is a component- based 3D representation of each building element (Winfield, 2015). BIM was introduced over a decade ago mainly to differentiate the information-rich
architectural 3D modeling from traditional 2D drawings, and it has been acclaimed
by its advocates as a lifesaver for complicated projects owing to its ability to correct
errors early in the design stage and accurately schedule construction (Kubba, 2012).
Recently, various BIM technology-based simulation tools have been developed for efficient
communication between the planning, design, construction, and maintenance phases of
a construction project and to ensure consistent project performance, but BIM has made
only a limited contribution to research in this field due to the limited interaction
between the real world and the virtual world (Chernick et al., 2020). However, integrating AR into BIM systems provides a platform on which to achieve
interactivity and can be helpful with regard to decision-making and efficiency of
processing due to its capacity for real-time visualization (Wang et al., 2014).
2.3 Augmented Reality
Augmented Reality (AR) refers to technologies and experiences that bring computer-generated
objects into the user’s physical (real-time) environment (Song et al., 2021). AR is a technology that enhances work efficiency by augmenting virtual information
in real time and allowing users to interact with augmented virtual information. Research
on AR began in the 1960s when van Surtherland developed the first see-through Head
Mounted Display (HMD), and in the early 1990s, Boeing coined the term ‘Augmented Reality’
(Korea Electronics and Telecommunications Research Institute). Generally, AR has become
increasingly relevant as a research field and the technology is expected to reach
a market size approaching 200 billion U.S. dollars by 2025 (Tatasciore, 2018).
Augmented reality uses a virtual environment created with computer graphics, but the
main character is the real environment. Computer graphics play a role in providing
the additional information necessary for the real environment. This means that the
real environment and the virtual screen are blurred by overlapping the 3D virtual
image on the actual image the user is viewing (Kim et al., 2013).
Behzadan and Kamat (2007) conducted a study to simulate a 3D object based on an image of a construction site,
and it is expected that the user’s visual recognition environment when accepting real
spatial information can be improved by using augmented reality. Currently, simultaneous
and multifaceted collaboration is required as construction projects become larger
and more complex from construction planning and design stages to maintenance (Jang et al., 2019). Therefore, it is necessary to check virtual buildings through AR technology and
to induce decision-making and high participation of building users through design
drawing revisions to achieve high performance.
3. Mesh Optimization Algorithm Approach
The mesh optimization algorithm was designed to create a low-poly version of a 3D
model effectively and accurately, proposing triangle combining and vertex merging
using reconstructions of common vertices for merged triangles. This type of algorithm
can merge three triangles into one triangle using the triangle centroid principle
via calculations that take the average of the x-coordinate, y-coordinate, and z-coordinate
of the triangle vertex (see Fig. 3).
Fig. 3. Steps of the Optimization Process and Concept
Our algorithm is based on the Unity C\# code based on the mesh finder and mesh level
of details (LOD) technique, which is capable of finding, merging and reconstructing
the triangular vertices of the 3D model using the triangular centroid calculation
principle composed of three points. We used the triangle center principle in the mesh
finder code, which counts the vertices and triangles of the model and draws a line
between the vertices and reconstructs three center points. In the mesh LOD code, codes
were written to add the ability to export the mesh code and divide the model into
whole/part models.
Unity, a cross-platform game engine developed by Unity Technologies, can be used to
create three-dimensional (3D) and two-dimensional (2D) games as well as interactive
simulations and other experiences and can be used for video applications, such as
those related to automobiles, architecture, engineering and construction (Xie, 2012). It has been adopted in industries other than gaming. To call it from Unity, you
need to attach a script to a Game Object (3D model) in your scene.
Scripts are written in a special language understandable by Unity, and the language
used by Unity is called C\#. The written code C\# in our algorithm is responsible
for finding, merging, and reconstructing the triangle vertices in the 3D model. For
the final step, we used a KNN to test the classification based on the 3D model vertex
distance dataset. A flow diagram of the proposed algorithm is shown in Fig. 4.
Fig. 4. Flowchart of the Proposed Algorithm
4. Experiment on the proposed mesh optimization algorithm
4.1 Specifications of the Test Model and 3D Modeling
A steel glass structure bridge with a length of 34 m, a height of 5 m, and a width
of 2 m connecting Sungkyunkwan University’s No.1 Engineering Building 23 and No. 2
Engineering Building 25 was selected as the target model for our algorithm. It was
modeled in Revit and passed to the Unity engine using the Unity Reflect process. Unity
Reflect for Autodesk Revit, industry-leading BIM software, enables designers and engineers
to transfer Revit models into real-time 3D experiences (2021 Unity Technologies).
It is possible to check all types of structure details as well as multiple meshes
and components (see Fig. 5).
Fig. 5. Target Model 3D Modeling
4.2 Data Collection and Processing Dataset using KNN Classification
KNN algorithms are used to find k-nearest neighbors of data points in a dataset, and
the basic method of finding k-nearest neighbors of a point is to compute all Euclidean
distances using data points (Xia et al., 2015). The dataset is collected by running the target 3D model multiple times with a similar
level of detail (Wang et al., 2021). We can determine the class to which a new input (test value) belongs when k-nearest
neighbors are chosen, and the distance is calculated between them. In our model, there
are thousands of combinations possible when merging vertices. Thus, we use the KNN
to determine the best fit to define the distance between the vertices. As a result
of extracting the distance dataset of points from the selection model using the unity
C\# code, we received a data set with 590,942 rows and four columns.
For data processing using KNN classification, the collected data mentioned earlier
are divided into a training dataset (80 %) and a test dataset (20 %) through the
Python package Scikit-learn. After the dataset train-test-split process, we used the
sklearn KNeighbors Classifier and KNN-predict to make predictions. Based on our collected
dataset, we obtained 90 % accuracy (see Fig. 6).
Fig. 6. Processing Dataset Using KNN Classification
4.3 Results and Discussion
As a result of our algorithm testing the SKKU steel bridge 3D model, although there
may be some difficulties in checking with the naked eye, as shown in Fig. 7, blue is the model before reduction and green is the model after reduction. Moreover,
there appears to be no problem in utilizing lightweight and original models for AR
visualization, as there are no significant differences visually.
Fig. 7. Result of BIM Model Mesh Optimization
In addition to the entire model before and after optimization, the number of points
and triangles before and after optimization for each structure could be checked immediately.
As shown in Table 1, the vertex of the original model can be reduced by approximately 56 % and the triangle
by about 42 %.
Table 1. Optimized Percentage Result of the Vertices and Triangles of the Model
$\dfrac{model\;Type}{Structure\;Type}$
|
Original model
|
Optimized model
|
Optimized %
$$
\left(V=\frac{V_{2}}{V_{1}} \times 100 \%, T=\frac{T_{2}}{T_{1}} \times
100 \%\right)
$$
|
Vertices 1
|
Triangles 1
|
Vertices 2
|
Triangles 2
|
Vertices
|
Triangles
|
Glass Structure
|
7214
|
4804
|
4405
|
1921
|
61 %
|
40 %
|
Tube Structure
|
20310
|
10438
|
10628
|
4174
|
52 %
|
40 %
|
Full model
|
590942
|
335564
|
330116
|
140167
|
56 %
|
42 %
|
5. Conclusion
Recently, with the introduction of augmented reality in the fourth industrial revolution
era, the construction industry is developing along with information and communication
technology. AR applied to the construction field is used during the process of reviewing
3D models for architectural designs and is also used for construction management at
construction sites.
In this paper, we proposed a new mesh reduction method based on central-triangular
vertex decimation combined with a smart mesh finder and mesh LOD in order to automate
the process of vertex merging by utilizing a KNN classification algorithm to optimize
the imported BIM model with meta-data from an external program to visualize AR effectively.
The classification operation well works on multi-hierarchy component 3D models and
meshes, allowing it easily to find the most necessary parts for optimization. Some
difficulty exists with regard to creating an easier import- export pipeline due to
various file formats and corresponding underlying systems. This newer approach improves
the topology preservation, and thus it is the best possible mesh reduction and reconstruction
strategy. This study aimed to optimize the imported BIM model with meta data from
an external program to visualize AR effectively, even on mobile devices with resource
constraints.
For future work, the trained model will be uploaded onto a dedicated server for greater
accessibility and availability. More diverse and complex mesh model testing will also
be conducted for more accuracy. In addition, the optimized BIM models will be tested
with an AR visualization device.
Acknowledgement
This research was conducted with the support of the “National R&D Project for Smart
Construction Technology (No.21SMIP- A158708-02)” funded by the Korea Agency for Infrastructure
Technology Advancement under the Ministry of Land, Infrastructure and Transport, and
managed by the Korea Expressway Corporation and supported by 「Innovative Talent Education
Program for Smart City」.
This manuscript is a revised version of the 2021 CONVENTION paper.
References
An J. Y., Choi J. M., Kwon S. H., Song D. H., Ock J. H. (2010). "A study on the development
of the project management system based on augmented reality.", Journal of the Korea
Academia-Industrial Cooperation Society, Vol. 11, No. 8, pp. 3083-3093 (in Korean)
Bahirat K., Lai C., Mcmahan R. P., Prabhakaran B. (2018). "Designing and evaluating
a mesh simplification algorithm for virtual reality.", ACM Transactions on Multimedia
Computing, Communications, and Applications (TOMM), Vol. 14, No. 3s, pp. 1-26
Behzadan A. H., Kamat V. R. (2007). "Georeferenced registration of construction graphics
in mobile outdoor augmented reality.", Journal of Computing in Civil Engineering,
Vol. 21, No. 4, pp. 247-258
Cao M. W., Li L., Xie W. J., Jia W., Lv Z. H., Zheng L. P., Liu X. P. (2019). "Parallel
K nearest neighbor matching for 3D reconstruction.", IEEE Access, Vol. 7, pp. 55248-55260
Chai C., Mustafa K., Kuppusamy S., Yusof A., Lim C. S., Wai S.H. (2019). "Bim integration
in augmented reality model.", International Journal of Technology, Vol. 10, No. 7,
pp. 1266-1275
Chernick A., Morse C., London S., Li T., Ménard D., Cerone J., Pasquarelli G. (2020).
"On-site BIM-enabled augmented reality for construction.", In The International Conference
on Computational Design and Robotic Fabrication, Springer, Singapore, pp. 46-56
Hoppe H., DeRose T., Duchamp T., McDonald J., Stuetzle W. (1993). "Mesh optimization.",
In Proceedings of the 20th Annual Conference on Computer Graphics and Interactive
Techniques, Anaheim, California, USA, pp. 19-26
Jang M. H., Choi H. B., Nam K. W. (2019). "Method to use drawings and 3D models in
building construction by augmented reality.", Journal of the Regional Association
of Architectural Institute of Korea, ISSN, Vol. 21, No. 6, pp. 1229-5752 (in Korean)
Karji A., Woldesenbet A., Rokooei S. (2017). "Integration of augmented reality, building
information modeling, and image processing in construction management: a content analysis.",
In AEI, pp. 983-992
Kim Y. T., Lee J. Y., Lee S. H., Choi J. S. (2013). "Construction and inspection management
system using mobile augmented reality.", In The 19th Korea-Japan Joint Workshop on
Frontiers of Computer Vision, IEEE, pp. 93-96
Kubba S. (2012). "Building information modeling, Handbook of Green Building Design
and Construction", pp. 201-206
Luebke D. P. (2001). "A developer’s survey of polygonal simplification algorithms.",
IEEE Computer Graphics and Applications, United States, Vol. 21, No. 3, pp. 24-35
Machado R. L., Vilela C. (2020). "Conceptual framework for integrating BIM and augmented
reality in construction management.", Journal of Civil Engineering and Management,
Vol. 26, No. 1, pp. 83-94
NBS Enterprises Ltd. (2020). "10th annual BIM report 2020", NBS, The Old Post Office,
St Nicholas Street, Newcastle upon Tyne, NE1 1RH, UK.
Peterson L. E. (2009). "K-nearest neighbor.", Scholarpedia, Vol. 4, No. 2, pp. 1883
Song Y., Koeck R., Luo S. (2021). "Review and analysis of augmented reality (AR) literature
for digital fabrication in architecture.", Automation in Construction, Vol. 128, pp.
103762
Tatasciore D. (2018). "DelivAR: An augmented reality mobile application to expedite
the package identification process for last-mile deliveries", Doctoral Dissertation,
Thesis for: M. Sc. Digital Media, University of Bremen, Available Online.
Wang B., Wu G., Zhao Q., Li Y., Gao Y., She J. (2021). "A topology-preserving simplification
method for 3D building models.", ISPRS International Journal of Geo-Information, Vol.
10, No. 6, pp. 422
Wang J., Wang X., Shou W., Xu B. (2014). "Integrating BIM and augmented reality for
interactive architectural visualisation.", Construction Innovation, Vol. 14, No. 4,
pp. 453-476
Winfield M. (2015). "Building information modelling: The legal frontier-overcoming
legal and contractual obstacles.", Society of Construction Law, pp. 18
Xia S., Xiong Z., Luo Y., Dong L., Zhang G. (2015). "Location difference of multiple
distances based k-nearest neighbors algorithm.", Knowledge-Based Systems, Vol. 90,
pp. 99-110
Xie J. (2012). "Research on key technologies base Unity 3D game engine.", International
Conference on Computer Science and Education (ICCSE), IEEE Access, pp. 695-699