Computing and Sharing the Differential Deformation of the Ground at a Continental Level Using Public EGMS Data ^†

Abstract

The European Ground Motion Service (EGMS) monitors and measures land displacement on a European scale using Sentinel-1 data, providing reliable and consistent data on natural ground motion phenomena. The Geomatics Research Unit of the Center Tecnològic de Telecomunicacions de Catalunya (CTTC) is working on a project to generate wide-area differential deformation maps from EGMS basic products and make this information available to the public through a web server. The project involves configuring a self-hosted, low-cost web server using open-source tools; adapting the ADAfinder application to identify active deformation areas (ADAs); developing software pipelines to compute and convert deformation data; and developing a tailored web visor to display the results. Automation is crucial to the project’s success since it must handle a significant volume of data with millions of PS points and long processing durations.

1. Introduction

The European Ground Motion Service (EGMS), part of the Copernicus Land Monitoring Service, detects and measures land displacement on a European scale. This service provides reliable and consistent information regarding both natural and anthropogenic hazards with a precision of a few millimeters and with annual updating [1]. The ground motion is derived from Sentinel-1 data using DInSAR techniques, PS (persistent scatterers), and DS (distributed scatterers) that contain mean annual velocities, displacement time series starting from 2015, line-of-sight information of both ascending and descending geometries, and horizontal and vertical components [1].

One of the current projects of the Geomatics Research Unit of the Center Tecnològic de Telecomunicacions de Catalunya (CTTC) aims to generate wide-area differential deformation maps that indicate the spatial gradient of the deformation field from EGMS basic products and then make this information available to the public through a map web and possibly a WMS or WMF layer. To achieve this goal, the authors have implemented various tasks to process the raw data delivered using the EGMS and thus enable the visualization of the final product through a web server.

The paper presents the said tasks related to algorithm and software development in a project that involves handling a vast amount of data. The project deals with 15,000 datasets, resulting in millions of PS points and extended processing times. Hence, automation is crucial to the project’s success.

2. Theory

Here, we present a semi-automatic tool to exploit the wide-area displacement maps of the EGMS with the final aim of identifying active deformation areas that may be at risk of damage. The potential risk of damage is based on the spatial gradients of deformation (differential deformation). In urban areas, having a map of the spatial gradient of deformation is crucial because most of the significant damages to manmade structures/infrastructures are associated with high deformation gradient values. In this section, first, the basic mathematical methods to determine the gradient of deformation and its orientation (slope and aspect) are presented and then the way to use the differential software tool and generate a differential deformation map from EGMS data is described. Several algorithms exist to calculate slopes, with the most common being the neighborhood method. This method calculates the slope at one grid point through comparing the values of the eight surrounding points in a 3 × 3 matrix [2]. Instead of using a rise-over-run calculation, Equation (1) assumes that a plane surface can be placed at any point on the surface z(x, y) in such a way that it only just touches the surface. In a 3 × 3 matrix of $\left[\begin{array}{ccc}{z}_1& z_2& z_3\\ z_4& z_5& z_6\\ z_7& z_8& z_9\end{array}\right]$, we can obtain $\frac{𝜕z}{𝜕x}$ and $\frac{𝜕z}{𝜕y}$ for the central grid ($z_5$) by dividing $\left[\begin{array}{c}{z}_1\\ 2z_4\\ z_7\end{array}\right]-\left[\begin{array}{c}{z}_3\\ 2z_6\\ z_9\end{array}\right]$ and $\left[\begin{array}{ccc}{z}_1& 2z_2& z_3\end{array}\right]-\left[\begin{array}{ccc}{z}_7& 2z_8& z_9\end{array}\right]$ into eight times the amount of grid spacing, respectively. Slope can be expressed in degrees or as a percentage, but our study specifically focuses on slope magnitude. Equation (2) relates to aspect calculation, which is the orientation of the slope measured clockwise in degrees from 0 to 360 where 0 is north-facing, 90 is east-facing, 180 is south-facing, and 270 is west-facing. An output aspect raster will typically result in several slope direction classes [2,3].

$$Slope=\sqrt{{\left(\frac{𝜕z}{𝜕x}\right)}^2+{\left(\frac{𝜕z}{𝜕y}\right)}^2}$$

(1)

$$Aspect=Arctan\frac{\left(dz/dx\right)}{\left(dz/dy\right)}$$

(2)

The main goal of this research is to calculate the slope and aspect starting from a land displacement map. This approach starts from the automatic extraction of the active deformation areas (ADAs) (to obtain more information look at [4] using the whole information inside a buffering area generated around each ADA. Note that points inside the ADA are active, while points outside the ADA (within the buffer) are not; it is crucial to have both types of points to perform a correct differential deformation analysis. To convert PS vector data to raster data, the suitable pixel size is defined to grid the entire ADA area; PS points inside each ADA polygon are interpolated by using the inverse distance weighted (IDW) technique [5]. Then, the neighborhood algorithm can be used to assess the magnitude of the slope and vector of an aspect through the PS’s raster format. Differential software tools generate two distinct shapefiles, one polygonal and one punctual, each one including the information related to slope and aspect, respectively. The differential deformation map reveals the slope intensity value to assign potential damage classes to ADAs. Moreover, monitoring the spatial variations of deformation can support the impact evaluation of motion phenomena, urban management, and planning activities.

3. Towards the Implementation

This section describes the steps already taken to implement a web to publish the differential deformation data for all of Europe.

3.1. The Server and Its Software Stack

Due to the size of the problem—a whole continent had to be processed with about 15,000 datasets in total—the first problem to solve was to decide where data and the WebGIS application itself would be stored. The first approach, using a cloud, was rejected because of the high cost it implied (thousands of euros per year). Therefore, an in-house solution was selected. An existing server was refurbished to host both data and applications. To set up the server and keep the cost low, the required software modules were selected from those available on the open-source (and free) market. These are as follows: (1) PostgreSQL plus PostGIS, to store geospatial data—that is, the points and ADAs; (2) GeoServer—a web-based tool to publish geospatial data—takes the information from a database, such as PostgreSQL, and makes it available through standard web-based protocols, such as WMS of WMF layers; (3) Apache + Tomcat—an httpd server plus a Java application server—to run both GeoServer and the web application itself. Finally, the web application is a self-developed tool programmed in JavaScript and HTML. It relies directly on GeoServer to retrieve the ADA data it displays.

3.2. The Data Production Pipeline

The points and polygons (ADAs), which are stored in the PostgreSQL database, are produced through a software pipeline. All their components but one (psql) have been developed by the authors. These are as follows: (1) ADAfinder—a tool in production since 2018. It takes the PSs from the EGMS and delivers ADAs. This tool was modified to specifically create ADAs with a buffer around their perimeter, so the computation of the differential movement was possible; (2) purge_overlaps. The EGMS delivers the PS in tiles that overlap. Due to this overlapping, ADAfinder may find repeated ADAs in the areas common to several tiles. With purge_overlaps, these are removed, keeping only a single instance. Note that this tool is necessary because of the way data is organized; if there was no overlapping, there would be no need to remove repeated ADAs; (3) The differential software tool takes the ADA polygons (with buffers), grids them into the appropriate pixel size, interpolates all PSs inside each polygon, and then computes the slope and aspect to finally create differential deformation maps of these areas; (4) point2PGIS and poly2PGIS—these tools take, respectively, the points and polygons (ADAs) created using the differential software tool mentioned above and produce a series of SQL-compliant files that include the same information but in a format that may be used to interact with the database. They process a single dataset at a time, which is a handicap; (5) To handle the vast volume of data, two tools, automate_point2PGIS and automate_poly2PGIS, were developed. These tools automate the process of running point2PGIS and poly2PGIS for each dataset, reducing processing time and minimizing human errors; (6) psql—this is a command-line tool bundled with PostgreSQL that is used to execute the files delivered through point2PGIS and poly2PGIS and thus insert the points and ADAs into the database. Note that as soon as a dataset produced through the software chain above is inserted into the database, it becomes automatically available to the web application. Figure 1 shows the workflow used to produce the ADAs and insert them into the database.

3.3. The Web Application

The web application has been developed in JavaScript and HTML and is able to visualize both the points and ADA polygons stored in the database. It can zoom, pan, and filter data using several criteria such as the mean velocity of the ADA or its quality assessment value. It is possible to use different background layers, such as OpenStreetMap, Google Satellite, or Google Terrain. To see the attributes of a point or an ADA, it is sufficient to click on it. Figure 2 depicts the web application. The legend and filters are in the upper right part of the image; the attribute table for points and ADAs is in the bottom right area.

4. Discussion, Conclusions, and Outlook

At the point of writing this paper, the server hardware and stack (Section 3.1) are complete. Synthetic datasets have been loaded to check the performance of the system and the results are satisfactory. The data production pipeline (Section 3.2) is only partially finished; the differential software tool is still under development, although it is already producing promising results. Additionally, no tool yet exists to automate its execution; this will have to wait until the differential software tool itself is completed. The remaining components of the pipeline are ready. Notably, in addition to devising and developing the algorithm computing the differential deformation of the terrain, it has been necessary to design a production workflow as well as automation tools to face the challenge of processing about 15,000 datasets alongside another one to deal with the problems created through the way data are organized—that of the overlapping tiles. This does not mean that solving how to compute the said differential deformation is a minor problem but that trying to tackle the challenge of computing it at a continental level creates other kinds of problems that must be solved if such a project is to be a success. Another problem to face when processing so much data is that of performance. All tools but the differential software tool have been streamlined to improve their performance; these have been checked with high volumes of synthetic data and the results show that there should be no bottlenecks. For instance, purge_overlaps can process data for the whole continent in about eight to ten hours; the insertion of SQL data into the database requires a similar time. It has to be mentioned that the differential software tool’s initial results covering the municipality of Barcelona are satisfactory. Due to the wide area of processing, this software requires considerable time investment in computations. As a result, minimizing processing time is a critical issue in optimizing the differential software tool. Furthermore, providing a new feature to filter PS points to remove a noisy PS that may refer to unwrapping errors is another factor in improving this software. This software relies on ADAfinder to generate results that cover all of Europe. To achieve this, ADA polygons for the entire continent within the EU must be created first. Fortunately, by now the initial results of the ADAfinder tool have been generated. After careful analysis and testing, the next step is to test the differential deformation software tool using ADAs from all over Europe.