DiMASH

When planning renovation and optimization measures for existing technical building systems, the initial stocktaking is a labor- and time-intensive task. With the DiMASH project, we aim to streamline this process by developing AI-based algorithms which scan images of technical building system diagrams and automatically transfer them into semantically enriched digital representations of the respective systems.

We use advanced computer vision and image processing methods to scan the depicted components, find the connecting lines, and read out descriptive text elements. The diagram components are extracted with a Faster R-CNN object detection network, trained and validated with a dataset of around 170 real technical building system diagrams, with a total number of approx. 22000 manually created labels, divided into 66 classes of components. The connecting lines and according junction points are found using image processing algorithms based on an anisotropic Gaussion filter. To detect and read out descriptive text elements, we use CRAFT Text Detector and Tesseract OCR (Optical Character Recognition).

Illustration of AI-based technical building system diagram scan pipeline
AI-based technical building system diagram scan pipeline, from left to right: (1) Original image of diagram, (2) Texts and component symbols detected and removed for subsequent line detection, (3) Found connected line segments and according junction points, (4) Reconstructed system topology combining all previously detected information.

The information destilled by the AI-based scan pipeline is used to digitally reconstruct the scanned schema and is structured with the TBSys (Technical Building System) Semantic Web ontology data model that we developed based on the ontologies Standard 223P and Brick Schema.

Overview of TBSys core concepts
TBSys (Technical Building System) ontology core classes, based on Standard 223P and Brick Schema.

This page hosts the demonstrator app that we build in the DiMASH project and which bundles the digitization process with downstream features for editing and extending the scanned contents. We develop and validate our approaches with a set of real technical building system diagrams from the building industry. For a final practical assessment of the overall procedure, we plan to apply the demonstrator app in two real construction projects.

We acknowledge the financial support by the German Federal Ministry for Economic Affairs and Climate Action in the project DiMASH (code 03EN1067A). More information can be found on the project website.

About Us

This site is provided by:

Fraunhofer Institute for Solar Energy Systems ISE
Group Building Performance Optimization
Team Cognitive Buildings
Heidenhofstraße 2
79110 Freiburg
Germany

Our group's research focus and topics are:

  • Concepts for building energy monitoring.
  • Analysis of the energy operation of buildings.
  • Development of automated fault detection and diagnosis (FDD) methods for building services.
  • Building Information Modelling (BIM) concepts for different phases of a building's lifecycle.

Contact