Development of advanced time series analysis techniques

The Working Group "Development of advanced time series analysis techniques" focuses on fundamental research for alternative data analysis techniques for challenging problems in climate and sustainability science. Data with gaps, irregular sampling, heavy tailed data, or uncertainties are exemplary challenges in the research at our institute. Recent developments offer alternative and advanced approaches for identifying the interactions in highly coupled complex systems, describing the stability of dynamical systems, or reducing the complexity of specific problems. These new concepts are further developed and application potentials are worked out. Together with the other research departments, these novel concepts are applied to specific climate impact problems.
 

Speaker: Norbert Marwan

Research background

Real world processes are complex and appear to be chaotic, i.e., their predictability is limited. Nonlinear theory offers new approaches for the development of modern data analysis techniques which can shed light on specific features in the dynamics of complex systems where standard or linear methods are not able to study them.

Example: Challenges in palaeoclimate research

Collecting water samples and measuring water parameters in the Waipuna cave, New Zealand, during fieldwork

Recent projects are collecting and extending the data archive of the palaeoclimate. New high precision records, e.g., such from stalagmites, offer new insights into the climate system or extreme events in the past that have not been available before on such high-resolution time-scales. A reliable interpretation requires statistical analysis and comparison with many environmental variables.

Moreover, such geological archives come with specific problems, such as time uncertainties and irregular sampling – challenges that hamper the application of standard methods or introducing nontrivial biases in the final results. Moreover, the definition of extreme climate events on the basis of geological archives is difficult, as it depends strongly on the time scale and since geological archives usually aggregate and integrate the environmental conditions over a longer period of time. There is a big need in methods which can be applied on such kind of data to derive reliable information about climate variability and the distribution of extremes while properly taking into account uncertainties.

Methods

Network representation of a multiplex recurrence network

We are developing and adopting methods of non-linear time series analysis for direct application to data with irregular sampling and with special focus on time uncertainties. This includes simple methods like kernel-based correlations, but also more complex methods like conditional mutual information, edit distance and recurrence based measures. We further develop methods for studying recurrence properties of extreme events data. New definitions of recurrence and extremes are in development for the particular consideration of time uncertainties. The recurrence concept is under extension to study spatio-temporal data. Moreover, we work on the development of methods for studying couplings, interrelations, and causal directions within extended systems. This research is also a basis for the reconstruction of complex networks and networks of networks.

Research Highlights

Development of time series analysis approaches and statistical association methods (correlation and mutual information) for the direct application on irregularly sampled time series. This development is the core for the subsequent investigation of regime changes and dynamical transitions, as well as spatio-temporal investigation of palaeo-climate data by complex networks (palaeo-climate networks).

 Solar impact Central America

Several approaches for the investigation of direct and indirect links, causal couplings, or coincidences of special events have been developed, critically tested, and finally applied to climate data, answering questions about differences in the spatial patterns of extreme monsoonal rainfall in India and South America, interrelations between the East Asian Summer monsoon and Indian Summer monsoon, or causal relationships between global warming, solar and volcanic activity, and human impact.

A time series with uncertainties can be considerd as a time series of probability density functions.

Introduction of a framework for dealing with time uncertainties in palaeo-climate proxy records (COPRA). This approach was the basis, e.g., for the discussion about the socio-economic climate impact in the Mayan civilization.

Characteristic and quantifiable features in recurrence plots

Further extending of recurrence analysis is an ongoing topic. The combination with the complex network approach has provided additional measures of complexity for the study of dynamical transitions in complex systems (recurrence networks). We have extended this approach by multiplex recurrence networks to study multivariate data, by new approaches for analysing irregularly sampled time series and data with uncertainties, by multiscale concepts, and by using alternative recurrence definitions for extreme events or spatio-temporal data. The critical and systematic study of the recurrence plot concept results in better understanding of this method and led to powerful correction schemes.

Toolbox for Complex Systems (TOCSY)

With Toolboxes for Complex Systems we provide a compilation of innovative methods for modern nonlinear data analysis. These methods were developed during our scientific research and cover recurrence analysis, causality investigations, filter procedures, time series analysis for irregularly sampled time series, etc.

Part of this toolbox are, e.g., the CRP Toolbox for MATLAB (providing many tools for recurrence analysis), or the pyunicorn package for Python (providing recurrence and complex network analysis tools).

Projects

IUCliD
Impacts of uncertainties in climate data analyses (IUCliD): Approaches to working with measurements as a series of probability distributions

September 2017 – 2021, Funded by: DFG - Deutsche Forschungsgemeinschaft
Contact: Marwan, Norbert

NatRiskChange
Natural Hazards and Risks in a Changing World

since October 2015, Funded by: DFG - Deutsche Forschungsgemeinschaft
Contact: Marwan, Norbert

Recurrence
Recurrence plot analysis of regime changes in dynamical Systems
December 2017 – March 2021, Funded by: DFG - Deutsche Forschungsgemeinschaft
Contact: Marwan, Norbert

QUEST
QUantitative paleoEnvironments from SpeleoThems
January 2016 – December 2019, Funded by: EU, H2020
Contact: Marwan, Norbert

NEMACS
Nonlinear Empirical Mode Analysis of Complex Systems: Development of General Approach and Applications in Climate
July 2019 –, Funded by: DFG - Deutsche Forschungsgemeinschaft
Contact: Marwan, Norbert

Trends, rhythms and events
Trends, rhythms and events in East African climate: statistical analysis of the paleoclimare records of the long sediment cores of the Chew Bahir basin
since 2017 –, Funded by: DFG - Deutsche Forschungsgemeinschaft
Contact: Marwan, Norbert

TiPES
Tipping points in the Earth system – Towards sharper estimates of critical forcing levels

since 2019, Funded by: EU, H2020
Contact: Marwan, Norbert

ClimXtreme
Klimawandel und Extremereignisse

since 2020, Funded by: BMBF
Contact: Marwan, Norbert

Research background

Real world processes are complex and appear to be chaotic, i.e., their predictability is limited. Nonlinear theory offers new approaches for the development of modern data analysis techniques which can shed light on specific features in the dynamics of complex systems where standard or linear methods are not able to study them.

Example: Challenges in palaeoclimate research

Collecting water samples and measuring water parameters in the Waipuna cave, New Zealand, during fieldwork

Recent projects are collecting and extending the data archive of the palaeoclimate. New high precision records, e.g., such from stalagmites, offer new insights into the climate system or extreme events in the past that have not been available before on such high-resolution time-scales. A reliable interpretation requires statistical analysis and comparison with many environmental variables.

Moreover, such geological archives come with specific problems, such as time uncertainties and irregular sampling – challenges that hamper the application of standard methods or introducing nontrivial biases in the final results. Moreover, the definition of extreme climate events on the basis of geological archives is difficult, as it depends strongly on the time scale and since geological archives usually aggregate and integrate the environmental conditions over a longer period of time. There is a big need in methods which can be applied on such kind of data to derive reliable information about climate variability and the distribution of extremes while properly taking into account uncertainties.

Methods

Network representation of a multiplex recurrence network

We are developing and adopting methods of non-linear time series analysis for direct application to data with irregular sampling and with special focus on time uncertainties. This includes simple methods like kernel-based correlations, but also more complex methods like conditional mutual information, edit distance and recurrence based measures. We further develop methods for studying recurrence properties of extreme events data. New definitions of recurrence and extremes are in development for the particular consideration of time uncertainties. The recurrence concept is under extension to study spatio-temporal data. Moreover, we work on the development of methods for studying couplings, interrelations, and causal directions within extended systems. This research is also a basis for the reconstruction of complex networks and networks of networks.

Research Highlights

Development of time series analysis approaches and statistical association methods (correlation and mutual information) for the direct application on irregularly sampled time series. This development is the core for the subsequent investigation of regime changes and dynamical transitions, as well as spatio-temporal investigation of palaeo-climate data by complex networks (palaeo-climate networks).

 Solar impact Central America

Several approaches for the investigation of direct and indirect links, causal couplings, or coincidences of special events have been developed, critically tested, and finally applied to climate data, answering questions about differences in the spatial patterns of extreme monsoonal rainfall in India and South America, interrelations between the East Asian Summer monsoon and Indian Summer monsoon, or causal relationships between global warming, solar and volcanic activity, and human impact.

A time series with uncertainties can be considerd as a time series of probability density functions.

Introduction of a framework for dealing with time uncertainties in palaeo-climate proxy records (COPRA). This approach was the basis, e.g., for the discussion about the socio-economic climate impact in the Mayan civilization.

Characteristic and quantifiable features in recurrence plots

Further extending of recurrence analysis is an ongoing topic. The combination with the complex network approach has provided additional measures of complexity for the study of dynamical transitions in complex systems (recurrence networks). We have extended this approach by multiplex recurrence networks to study multivariate data, by new approaches for analysing irregularly sampled time series and data with uncertainties, by multiscale concepts, and by using alternative recurrence definitions for extreme events or spatio-temporal data. The critical and systematic study of the recurrence plot concept results in better understanding of this method and led to powerful correction schemes.

Toolbox for Complex Systems (TOCSY)

With Toolboxes for Complex Systems we provide a compilation of innovative methods for modern nonlinear data analysis. These methods were developed during our scientific research and cover recurrence analysis, causality investigations, filter procedures, time series analysis for irregularly sampled time series, etc.

Part of this toolbox are, e.g., the CRP Toolbox for MATLAB (providing many tools for recurrence analysis), or the pyunicorn package for Python (providing recurrence and complex network analysis tools).

Projects

IUCliD
Impacts of uncertainties in climate data analyses (IUCliD): Approaches to working with measurements as a series of probability distributions

September 2017 – 2021, Funded by: DFG - Deutsche Forschungsgemeinschaft
Contact: Marwan, Norbert

NatRiskChange
Natural Hazards and Risks in a Changing World

since October 2015, Funded by: DFG - Deutsche Forschungsgemeinschaft
Contact: Marwan, Norbert

Recurrence
Recurrence plot analysis of regime changes in dynamical Systems
December 2017 – March 2021, Funded by: DFG - Deutsche Forschungsgemeinschaft
Contact: Marwan, Norbert

QUEST
QUantitative paleoEnvironments from SpeleoThems
January 2016 – December 2019, Funded by: EU, H2020
Contact: Marwan, Norbert

NEMACS
Nonlinear Empirical Mode Analysis of Complex Systems: Development of General Approach and Applications in Climate
July 2019 –, Funded by: DFG - Deutsche Forschungsgemeinschaft
Contact: Marwan, Norbert

Trends, rhythms and events
Trends, rhythms and events in East African climate: statistical analysis of the paleoclimare records of the long sediment cores of the Chew Bahir basin
since 2017 –, Funded by: DFG - Deutsche Forschungsgemeinschaft
Contact: Marwan, Norbert

TiPES
Tipping points in the Earth system – Towards sharper estimates of critical forcing levels

since 2019, Funded by: EU, H2020
Contact: Marwan, Norbert

ClimXtreme
Klimawandel und Extremereignisse

since 2020, Funded by: BMBF
Contact: Marwan, Norbert