In this article, we provide an overview of what has been developed in the frame of BIMprove from the perspective of the final users, our industrial partners. Most functions of the BIMprove system have reached a prototype stage by M24 (as planned) and testing of the overall system has been started on the three Pilot Use-Case locations.
BIMprove components
BIMprove is a project which aim is to contribute with a solution for monitoring a construction site using digital data, in order to improve and automate the follow-up of the quality, scheduling and safety of the site. In short, BIMprove is able to compare and analyse a BIM model and different type of real-time data from the construction site, in order to create a digital twin of the worksite. The work carried out in the frame of BIMprove resulted in development of hardware, software as well as methods that are described in the following sections:
Hardware components
Objective | Title | Description |
Data capture | UGV to take photos, thermal pictures and 3D scanning | For capturing visual data, we have further developed a ground robot prototype that can, both, visit the worksite capturing: pictures, 3D scan (to provide point-clouds) and thermal pictures. The first integration of Leica BLK360 in UGV application. Autonomous navigation based on planned mission. |
Data capture | UAV to take photos, thermal pictures and 3D scanning | For capturing visual data, we have developed a drone prototype that can both, visit the worksite and capture: pictures, 3D scans (to provide point-clouds) and thermal images. The path planning is based on .ifc data that is converted to an Octomap (3D-occupancy grid). So, the drone can navigate fully autonomous on a worksite, an onboard depth camera detects obstacles and adds them to the octomap. A modified A* algorithm is implemented for the pathplanning. |
Service | Controller for UGV | For being able to control the robot remotely and launch autonomous functionalities, we have been working in a 2.4gHz WiFi based new connection with a greater gain antenna so that this connection can be established from a greater distance. The command service can be used with a remote control and a tablet-like device. |
Service | Base-station for UAV | The drone flying indoor does not have an absolute position and orientation. To solve this problem, we developed a landing base for the drone, that is also modelled and stored in the BIM. With the help of this, the drone is always located in the correct frame and on-board measuring devices are aligned with the global frame. Secondly the base station has charging capabilities such that the drone is recharged automatically after landing. |
Data capture | Worker’s presence detection | Activity, in terms of presence of number of workers, in all working zones can be logged, and it can be live shown in BIMsync with a compatible model. There are two central hardware elements in this system:
To support and maintain reliable zone association of workers, a software system for reliable tracking based on a zone connectivity graph have been developed. For the sake of scalability to large sites with hundreds to thousands of zones, a distributed system architecture has been chosen. |
Methods
Objective | Title | Description |
Processing point clouds | Automatic alignment of the point cloud with the IFC coordinates system | A clear workflow was developed to align measurements with the BIM model. Functionality of given software (Pix4D, Leica Cyclone) was extended in a way such the alignment of the model and the measured data needs very few interactions by the user. It uses the newly developed markers that are standardised in the CEN workshop agreement.
In the case of Pix4D pointclouds, the process is fully automated, in the case of the Leica cyclone software few interactions (aligning BW-markers to specific points) is needed. |
Comparing point-cloud with IFC model | Automatic comparing process of geometries from point clouds and IFC models | Automatic comparing process between the geometry of the BIM model and the point-cloud. Input is a BIM-model and a point cloud, output is a BCF issue describing the result. The end user considers this and makes a decision. |
Comparing point-cloud with IFC model | Automatic comparing process of IFC elements existing on the point cloud and the executed due date of each element | Based on a BCF issue with links to the model and the comparison part of the point cloud, the user can as part of the daily decision process decide that work is delayed (or in theory ahead of time). If so, the user can assign the issue to the scheduler (if this is a different role) and as that the task covering this in the schedule is updated. |
Standardization | Use of targets during scanning to align pointcloud with the local coordinates system | Definition of Markers with machine readable coordinates which can be measured by surveyors. The markers enable an automated alignment of the point cloud with precise coordinates both in real-time and in post processing. Tags for VR glasses or drone orientation can also be part of the markers. The definition of the markers is developed in a CEN Workshop, a pre-standardization initiative by CEN which results in a so-called CEN Workshop Agreement (CWA) and aims to be a future standard. |
AI for pictures | Image recognition | Risk Object Visual Analysis System (ROVAS) for automatic detection of safety measures, especially safety nets, from photographs and 3D point clouds captured at construction sites. |
Decision | Daily Decision Cycle | A schedule and a set of BIM models exists and are kept up to date. Each day the BIMprove operator looks at tasks that are supposed to be completed today and creates a “high level scan request” to the robot and/or drone operators. They scan and send back the results using BCF. The decision maker (a role) looks at the scans and decides if the conclusion is (1) Good result, accept (w) Not enough info, scan again, (3) send a person to check, (4) Done, but not perfect quality – rework must be done and scheduled, (5) Done, significant difference from the plan, but might be to expensive to correct – ask client if it is acceptable together with other compensation. |
Picture | Pictures including Exif data field | Embedded Exif field with position and pose of the camera for machine learning and image classification |
Software functionalities
Objective | Title | Description |
Data capture | Definition of a capture mission with BCF | Drone: A dedicated application has been developed such that a semi-automatic data capturing workflow is executed. It first starts with the user defining a waypoint as a BCF issue for a drone scan, then the application obtaining that specific waypoint and sending it to the drone for a mission. All data captured will be transferred through the workstation via the application automatically after the drone is done with the mission. Then, thermal data post-processing will begin to analyse the captured data whether there is a risk in term of thermal issue or not. If yes, the application will create a BCF issue and upload/link all the problematic data to the specific BCF issue created before sending out an alert via SMS to the dedicated numbers or responsible persons. |
4D viewer | Create BCF issue board from MS Project | An MS Project file can be imported and based on this a BCF issue board is created. The benefit is that BCF is an open standard and that changing it is easy allowing the schedule to be more dynamic and accessible. BCF also have many nice attributes, like being made for linking with the BIM model. |
4D viewer | 4D simulation viewer linked with IFC | When the BCF-backed schedule tasks are linked with BIM-objects, playing back the schedule is possible with a timeline animation. When the date where a task is worked on is shown, the 3D viewer shows the related objects appearing. |
4D viewer | 3D/4D viewer of delays | Comparation between the geometry of the point cloud and the BIM model (plan vs scan) is part of the consideration when updating the schedule and capturing delays and measures taken. |
3D viewer | Workers’ presence viewer | Heatmap visualizing the position of workers with wearables. Workers use wearable trackers, and we can visualize in 3D how many people are at each location. To protect privacy, only the number of workers per zone is visualized. |
Pointcloud processing | Pointcloud comparison (IfcEntity ↔︎ Pointcloud) | pyBIMprove compares each IfcEntities with data from a point cloud. The entities to be compared against can be controlled through a selection of GUID’S. Comparison works by trying to minimize point distance to mesh surface (best fit), resulting in a offset for the placement if the ifcEntity. From the comparison there is generated several filtered coloured point clouds, where the colour is either natural or coloured as a heatmap. There is also defined a feature set to try to estimate the success of the comparison, this result can be extract as a pure JSON-format. Here are some example of heat maps, from Fyrstikkbakken 14 BC, 9th floor.
|
Visual inspections | Virtual reality | Before the project FhG-IAO and USTUTT had already developed a single-user VR software called XR-Visualizer. Its main feature was the easy loading and integration of 3D-models into VR. This software was built upon and greatly improved throughout the BIMprove project. The BIMprove XR Viewer is now a Multi-user Virtual Reality* environment to visualize IFC files and point clouds as well as communicate with BCF files (marking, issue creation). It has been installed and tested at the PUCs in Lausanne Bussigny and Madrid. (*There is also desktop-PC-version with which to join (VR-)sessions.) |
Visual inspections | Augmented reality | BIM@Construction AR tool allows user to visualize the BIM model at the construction site. The main features of AR-tool has been developed and improved during BIMprove. Main development environment has been Unity3D and it could be exploited in Microsoft HoloLens 2 and high-end android tablet. AR tool includes several tools, which user can use in 1st person mode, like notes, measure tools, warning signs etc. BIM@Construction has been tested at the PUCs in Lausanne Bussigny and demonstrated in Madrid. As tool is proof of concept it could be future developed and provide Software as a Service.
|
Image processing | Software service for detecting safety elements on pictures | Machine learning based computer vision model is trained for detecting risks or risk countermeasures at the construction sites. The model’s detection capabilities are provided as a service through a suitable REST interface. It can load the model and uses it to analyse the images send to it by HTTP POST. The detection results are returned as JSON text and described in the deliverable D2.4 Detailed Description of the Safety Functionalities and Worker Notifications. |
Image processing | Picture + detection → IfcElement in Safety model | ![]() The starting point is a geometry model and the definition of safety elements (here: barriers). Data acquisition with UGV/UAV is actually intended for geometry surveying. However, the data can also be processed with the ROVAS system and additionally with additional information (where and at what angle was the image taken). Together with the detection of e.g. barriers, this can now be compared with the safety model and checked whether these facilities are present where required. |
3D viewer | Viewer of photos with thermal images | Based on the stored Exif data in the taken thermal pictures, the system can automatically present on the BIM model, where the photo was taken and from which angle.
|
Visual inspections | Model & point cloud delivery for XR | Semi-automised conversion of IFC- and point cloud files for VR and AR. The software consists of scripts for the semi-automised utilisation and parameterisation of pre-existent third party software. As of now, its utilisation can be offered as a service by FhG-IAO. The workflow has been described in earlier Deliverables of the project such as D2.7, D2.8, and D3.1. |
User interface | Human-Machine Interface for UGV | The original HMI for the Summit-XL mobile ground robot has been extended with new embedded functions. Such functions cover the control of 3D scanner, autonomy properties and path following capabilities. |
3D viewer | Path-planning for UGV | An automated tool to generate the shortest path in an IFC model from the current position to an emergency exit is developed.
|
Data capture | Capability for automatic scan with UGV | The Summit-XL mobile ground robot has been extended and integrated with RS-Lidar 16, Flir Ax8 and BLK 360. The integration description can be found in D3.4. Prototype System Description and Test Results. |
BIMprove Benchmarking
In this section, we analyse the 6 objectives set out at the beginning of the project (the initial proposal) with the implementation results achieved at the end of the project.
Do you want to learn more about BIMprove results?
Read here our latest Deliverable 3.6 Benchmarking, Evaluation & Demonstration!
Visit our Website, subscribe to our YouTube channel and share your opinion with us through our LinkedIn or Twitter communities!