Botache, D.; Decke, J.; Ripken, W.; Dornipati, A.; Götz-Hahn, F.; Ayeb, M. & Sick, B. (2024),
'Enhancing Multi-Objective Optimization through Machine Learning-Supported Multiphysics Simulation', arXiv e-prints
, arXiv:2309.13179v2
.
[BibTeX]
[Endnote]
This paper presents a methodological framework for training, self-optimising, and self-organising surrogate models to approximate and speed up multiobjective optimisation of technical systems based on multiphysics simulations. At the hand of two real-world datasets, we illustrate that surrogate models can be trained on relatively small amounts of data to approximate the underlying simulations accurately. Including explainable AI techniques allow for highlighting feature relevancy or dependencies and supporting the possible extension of the used datasets. One of the datasets was created for this paper and is made publicly available for the broader scientific community. Extensive experiments combine four machine learning and deep learning algorithms with an evolutionary optimisation algorithm. The performance of the combined training and optimisation pipeline is evaluated by verifying the generated Pareto-optimal results using the ground truth simulations. The results from our pipeline and a comprehensive evaluation strategy show the potential for efficiently acquiring solution candidates in multiobjective optimisation tasks by reducing the number of simulations and conserving a higher prediction accuracy, i.e., with a MAPE score under 5% for one of the presented use cases.
Decke, J.; Jenß, A.; Sick, B. & Gruhl, C. (2024),
An Efficient Multi Quantile Regression Network with Ad Hoc Prevention of Quantile Crossing, in
'International Conference on Architecture of Computing Systems (ARCS)'
, Springer,
.
[BibTeX]
[Endnote]
This article presents the Sorting Composite Quantile Regression Neural Network (SCQRNN), an advanced quantile regression model designed to prevent quantile crossing and enhance computational efficiency. Integrating ad hoc sorting in training, the SCQRNN ensures non-intersecting quantiles, boosting model reliability and interpretability. We demonstrate that the SCQRNN not only prevents quantile crossing and reduces computational complexity but also achieves faster convergence than traditional models. This advancement meets the requirements of high-performance computing for sustainable, accurate computation. In organic computing, the SCQRNN enhances self-aware systems with predictive uncertainties, enriching applications across finance, meteorology, climate science, and engineering.
Decke, J.; Wünsch, O.; Sick, B. & Gruhl, C. (2024),
From Structured to Unstructured: A Comparative Analysis of Computer Vision and Graph Models in solving Mesh-based PDEs, in
'International Conference on Architecture of Computing Systems (ARCS)'
, Springer,
.
[BibTeX]
[Endnote]
This article investigates the application of computer vision and graph-based models in solving mesh-based partial differential equations within high-performance computing environments. Focusing on structured, graded structured, and unstructured meshes, the study compares the performance and computational efficiency of three computer vision-based models against three graph-based models across three datasets. The research aims to identify the most suitable models for different mesh topographies, particularly highlighting the exploration of graded meshes, a less studied area. Results demonstrate that computer vision-based models, notably U-Net, outperform the graph models in prediction performance and efficiency in two (structured and graded) out of three mesh topographies. The study also reveals the unexpected effectiveness of computer vision-based models in handling unstructured meshes, suggesting a potential shift in methodological approaches for data-driven partial differential equation learning. The article underscores deep learning as a viable and potentially sustainable way to enhance traditional high-performance computing methods, advocating for informed model selection based on the topography of the mesh.
Heidecker, F.; El-Khateeb, A.; Bieshaar, M. & Sick, B. (2024),
'Criteria for Uncertainty-based Corner Cases Detection in Instance Segmentation', arXiv e-prints
, arXiv:2404.11266
.
[BibTeX]
[Endnote]
The operating environment of a highly automated vehicle is subject to change, e.g., weather, illumination, or the scenario containing different objects and other participants in which the highly automated vehicle has to navigate its passengers safely. These situations must be considered when developing and validating highly automated driving functions. This already poses a problem for training and evaluating deep learning models because without the costly labeling of thousands of recordings, not knowing whether the data contains relevant, interesting data for further model training, it is a guess under which conditions and situations the model performs poorly. For this purpose, we present corner case criteria based on the predictive uncertainty. With our corner case criteria, we are able to detect uncertainty-based corner cases of an object instance segmentation model without relying on ground truth (GT) data. We evaluated each corner case criterion using the COCO and the NuImages dataset to analyze the potential of our approach. We also provide a corner case decision function that allows us to distinguish each object into True Positive (TP), localization and/or classification corner case, or False Positive (FP). We also present our first results of an iterative training cycle that outperforms the baseline and where the data added to the training dataset is selected based on the corner case decision function.
Magnussen, B. M.; Möckel, F.; Jessulat, M.; Stern, C. & Sick, B. (2024),
Optical Detection of the Body Mass Index and Related Parameters Using Multiple Spatially Resolved Reflection Spectroscopy, in
'International Conference on Bioinformatics and Computational Biology (ICBCB)'
, IEEE,
.
[BibTeX]
[Endnote]
Obesity is a common problem for many people. In order to assist people combating obesity, providing methods to quickly, easily, and inexpensively assess their body composition is important. This article investigates how noninvasive, optical sensors based on multiple spatially resolved reflection spectroscopy can be used to measure the body mass index and body composition parameters. Using machine learning to train continuous feature networks, it is possible to predict the body mass index of a subject, with a correlation of $R=0.61$ with $p<0.0001$. Similarly, the predicted body mass index shows correlations to both the subject's visceral fat ($R=0.44,p=0.0023$) and skeletal muscle mass index ($R=0.52,p=0.0003$), indicating that the trained neural network is capable of identifying both types of tissue. Strategies to independently detect either type of tissue are discussed.
Aßenmacher, M.; Rauch, L.; Goschenhofer, J.; Stephan, A.; Bischl, B.; Roth, B. & Sick, B. (2023),
Towards Enhancing Deep Active Learning with Weak Supervision and Constrained Clustering, in
'Workshop on Interactive Adapative Learning (IAL), ECML PKDD'
, pp. 65--73
.
[BibTeX]
[Endnote]
Three fields revolving around the question of how to cope with limited amounts of labeled data are Deep Active Learning (DAL), deep Constrained Clustering (CC), and Weakly Supervised Learning (WSL). DAL tackles the problem by adaptively posing the question of which data samples to annotate next in order to achieve the best incremental learning improvement, although it suffers from several limitations that hinder its deployment in practical settings. We point out how CC algorithms and WSL could be employed to overcome these limitations and increase the practical applicability of DAL research. Specifically, we discuss the opportunities to use the class discovery capabilities of CC and the possibility of further reducing human annotation efforts by utilizing WSL. We argue that the practical applicability of DAL algorithms will benefit from employing CC and WSL methods for the learning and labeling process. We inspect the overlaps between the three research areas and identify relevant and exciting research questions at the intersection of these areas.
Breitenstein, J.; Heidecker, F.; Lyssenko, M.; Bogdoll, D.; Bieshaar, M.; Zöllner, J. M.; Sick, B. & Fingscheidt, T. (2023),
What Does Really Count? Estimating Relevance of Corner Cases for Semantic Segmentation in Automated Driving, in
'Workshop on roBustness and Reliability of Autonomous Vehicles in the Open-world (BRAVO), ICCV'
, pp. 3991--4000
.
[BibTeX]
[Endnote]
In safety-critical applications such as automated driving, perception errors may create an imminent risk to vulnerable road users (VRU). To mitigate the occurrence of unexpected and potentially dangerous situations, so-called corner cases, perception models are trained on a huge amount of data. However, the models are typically evaluated using task-agnostic metrics, which do not reflect the severity of safety-critical misdetections. Consequently, misdetections with particular relevance for the safe driving task should entail a more severe penalty during evaluation to pinpoint corner cases in large-scale datasets. In this work, we propose a novel metric IoUw that exploits relevance on the pixel level of the semantic segmentation output to extend the notion of the intersection over union (IoU) by emphasizing small areas of an image affected by corner cases. We (i) employ IoUw to measure the effect of pre-defined relevance criteria on the segmentation evaluation, and (ii) use the relevance-adapted IoUw to refine the identification of corner cases. In our experiments, we investigate vision-based relevance criteria and physical attributes as per-pixel criticality to factor in the imminent risk, showing that IoUw precisely accentuates the relevance of corner cases.
Decke, J.; Gruhl, C.; Rauch, L. & Sick, B. (2023),
DADO – Low-Cost Query Strategies for Deep Active Design Optimization, in
'International Conference on Machine Learning and Applications (ICMLA)'
, IEEE,
, pp. 1611--1618
.
[BibTeX]
[Endnote]
In this work, we apply deep active learning to the field of design optimization to reduce the number of computationally expensive numerical simulations widely used in industry and engineering. We are interested in optimizing the design of structural components, where a set of parameters describes the shape. If we can predict the performance based on these parameters and consider only the promising candidates for simulation, there is an enormous potential for saving computing power. We present two query strategies for self-optimization to reduce the computational cost in multi-objective design optimization problems. Our proposed methodology provides an intuitive approach that is easy to apply, offers significant improvements over random sampling, and circumvents the need for uncertainty estimation. We evaluate our strategies on a large dataset from the domain of fluid dynamics and introduce two new evaluation metrics to determine the model's performance. Findings from our evaluation highlights the effectiveness of our query strategies in accelerating design optimization. Furthermore, the introduced method is easily transferable to other self-optimization problems in industry and engineering.
Deinzer, R.; Eidenhardt, Z.; Sohrabi, K.; Stenger, M.; Kraft, D.; Sick, B.; Götz-Hahn, F.; Bottenbruch, C.; Berneburg, N. & Weik, U. (2023),
'It is the habit not the handle that affects tooth brushing. Results of a randomised counterbalanced cross over study', Research Square
.
[BibTeX]
[Endnote]
Objective: To assess the effect of the toothbrush handle on video-observed toothbrushing behaviour and toothbrushing effectiveness Methods: This is a randomized counterbalanced cross-over study. N = 50 university students brushed their teeth at two occasions, one week apart, using either a commercial ergonomically designed manual toothbrush (MT) or Brushalyze V1 (BV1), a manual toothbrush with a thick cylindrical handle without any specific ergonomic features. Brushing behaviour was video-analysed. Plaque was assessed at the second occasion immediately after brushing. Participants directly compared the two brushes regarding their handling and compared them to the brushed they used at home. Results: The study participants found the BV1 significantly more cumbersome than the M1 or their brush at home. (p < 0.05). However, correlation analyses revealed a strong consistency of brushing behavior with the two brushes (0.71 < r < 0.91). Means differed only slightly (all d < 0.36). These differences became statistically significant only for the brushing time at inner surfaces (d = 0.31 p = 0.03) and horizontal movements at inner surfaces (d = 0.35, p = 0.02). Plaque levels at the gingival margins did not differ while slightly more plaque persisted at the more coronal aspects of the crown after brushing with BV1 (d = 0.592; p 0.042) Discussion: The results of the study indicate that the brushing handle does not play a major role in brushing behavior or brushing effectiveness.
Heidecker, F.; Susetzky, T.; Fuchs, E. & Sick, B. (2023),
Context Information for Corner Case Detection in Highly Automated Driving, in
'IEEE International Conference on Intelligent Transportation Systems (ITSC)'
, IEEE,
, pp. 1522--1529
.
[BibTeX]
[Endnote]
Context information provided along with a dataset can be very helpful for solving a problem because the additional knowledge is already available and does not need to be extracted. Moreover, the context indicates how diverse a dataset is, i.e., how many samples per context category are available to train and test machine learning (ML) models. In this article, we present context annotations for the BDD100k image dataset. The annotations comprise, for instance, information about daytime, road condition (dry/wet), and dirt on the windshield. Sometimes, no or only little data are available for unique or rare combinations of these context attributes. However, data that matches these context conditions is crucial when discussing corner cases: Firstly, most ML models, e.g., object detectors, are not trained on such data, which leads to the assumption that they will perform poorly in those situations. Secondly, data containing corner cases are required for validating ML models. With this in mind, separate ML models dedicated to context detection are useful for expanding the training set with additional data of special interest, such as corner cases.
Heidecker, F.; Bieshaar, M. & Sick, B. (2023),
'Corner Cases in Machine Learning Processes', AI Perspectives & Advances
6
(1)
, 1--17
.
[BibTeX]
[Endnote]
Applications using machine learning (ML), such as highly autonomous driving, depend highly on the performance of the ML model. The data amount and quality used for model training and validation are crucial. If the model cannot detect and interpret a new, rare, or perhaps dangerous situation, often referred to as a corner case, we will likely blame the data for not being good enough or too small in number. However, the implemented ML model and its associated architecture also influence the behavior. Therefore, the occurrence of prediction errors resulting from the ML model itself is not surprising. This work addresses a corner case definition from an ML model's perspective to determine which aspects must be considered. To achieve this goal, we present an overview of properties for corner cases that are beneficial for the description, explanation, reproduction, or synthetic generation of corner cases. To define ML corner cases, we review different considerations in the literature and summarize them in a general description and mathematical formulation, whereby the expected relevance-weighted loss is the key to distinguishing corner cases from common data. Moreover, we show how to operationalize the corner case characteristics to determine the value of a corner case. To conclude, we present the extended taxonomy for ML corner cases by adding the input, model, and deployment levels, considering the influence of the corner case properties.
Huang, Z.; He, Y. & Sick, B. (2023),
Spatio-Temporal Attention Graph Neural Network for Remaining Useful Life Prediction, in
'Computational Science and Computational Intelligence (CSCI)'
, IEEE,
.
[BibTeX]
[Endnote]
RUL prediction plays a crucial role in the health management of industrial systems. Given the increasing complexity of systems, data-driven predictive models have attracted significant research interest. Upon reviewing the existing literature, it appears that many studies either do not fully integrate both spatial and temporal features or employ only a single attention mechanism. Furthermore, there seems to be inconsistency in the choice of data normalization methods, particularly concerning operating conditions, which might influence predictive performance. To bridge these observations, this study presents the Spatio-Temporal Attention Graph Neural Network. Our model combines graph neural networks and temporal convolutional neural networks for spatial and temporal feature extraction, respectively. The cascade of these extractors, combined with multihead attention mechanisms for both spatio-temporal dimensions, aims to improve predictive precision and refine model explainability. Comprehensive experiments were conducted on the CMAPSS dataset to evaluate the impact of unified versus clustering normalization. The findings suggest that our model performs state-of-the-art results using only the unified normalization. Additionally, when dealing with datasets with multiple operating conditions, cluster normalization enhances the performance of our proposed model by up to 27%.
Magnussen, B. M.; Stern, C. & Sick, B. (2023),
'Continuous Feature Networks: A Novel Method to Process Irregularly and Inconsistently Sampled Data With Position-Dependent Features', International Journal On Advances in Intelligent Systems
16
(3&4)
, 43--50
.
[BibTeX]
[Endnote]
Continuous Kernels have been a recent development in convolutional neural networks. Such kernels are used to process data sampled at different resolutions as well as irregularly and inconsistently sampled data. Convolutional neural networks have the property of translational invariance (e.g., features are detected regardless of their position in the measurement domain), which is unsuitable if the position of detected features is relevant for the prediction task. However, the capabilities of continuous kernels to process irregularly sampled data are still desired. This article introduces the continuous feature network, a novel method utilizing continuous kernels, for detecting global features at absolute positions in the data domain. Through a use case in processing multiple spatially resolved reflection spectroscopy data, which is sampled irregularly and inconsistently, we show that the proposed method is capable of processing such data directly without additional preprocessing or augmentation as is needed using comparable methods. In addition, we show that the proposed method is able to achieve a higher prediction accuracy than a comparable network on a dataset with position-dependent features. Furthermore, a higher robustness to missing data compared to a benchmark network using data interpolation is observed, which allows the network to adapt to sensors with a failure of individual light emitters or detectors without the need for retraining. The article shows how these capabilities stem from the continuous kernels used and how the number of available kernels to be trained affects the model. Finally, the article proposes a method to utilize the introduced method as a base for an interpretable model usable for explainable AI.
Magnussen, B. M.; Stern, C. & Sick, B. (2023),
Leveraging Repeated Unlabelled Noisy Measurements to Augment Supervised Learning, in
'International Conference on Computational Intelligence and Intelligent Systems (CIIS)'
, ACM,
, pp. 1--6
.
[BibTeX]
[Endnote]
Often, producing large labelled datasets for supervised machine learning is difficult and expensive. In cases where the expensive part is due to labelling and obtaining ground truth, it is often comparably easy to acquire large datasets containing unlabelled data points. For reproducible measurements, it is possible to record information on multiple data points being from the same reproducible measurement series, which should thus have an equal but unknown ground truth. In this article, we propose a method to incorporate a dataset of such unlabelled data points for which some data points are known to be equal in end-to-end training of otherwise labelled data. We show that, with the example of predicting the carotenoid concentration in human skin from optical multiple spatially resolved reflection spectroscopy data, the proposed method is capable of reducing the required number of labelled data points to achieve the same prediction accuracy for different model architectures. In addition, we show that the proposed method is capable of reducing the negative impact of noisy data when performing a repeated measurement of the same sample.
Nivarthi, C. P.; Vogt, S. & Sick, B. (2023),
'Multi-Task Representation Learning for Renewable-Power Forecasting: A Comparative Analysis of Unified Autoencoder Variants and Task-Embedding Dimensions', Machine Learning and Knowledge Extraction (MAKE)
5
(3)
, 1214--1233
.
[BibTeX]
[Endnote]
Typically, renewable-power-generation forecasting using machine learning involves creating separate models for each photovoltaic or wind park, known as single-task learning models. However, transfer learning has gained popularity in recent years, as it allows for the transfer of knowledge from source parks to target parks. Nevertheless, determining the most similar source park(s) for transfer learning can be challenging, particularly when the target park has limited or no historical data samples. To address this issue, we propose a multi-task learning architecture that employs a Unified Autoencoder (UAE) to initially learn a common representation of input weather features among tasks and then utilizes a Task-Embedding layer in a Neural Network (TENN) to learn task-specific information. This proposed UAE-TENN architecture can be easily extended to new parks with or without historical data. We evaluate the performance of our proposed architecture and compare it to single-task learning models on six photovoltaic and wind farm datasets consisting of a total of 529 parks. Our results show that the UAE-TENN architecture significantly improves power-forecasting performance by 10 to 19% for photovoltaic parks and 5 to 15% for wind parks compared to baseline models. We also demonstrate that UAE-TENN improves forecast accuracy for a new park by 19% for photovoltaic parks, even in a zero-shot learning scenario where there is no historical data. Additionally, we propose variants of the Unified Autoencoder with convolutional and LSTM layers, compare their performance, and provide a comparison among architectures with different numbers of task-embedding dimensions. Finally, we demonstrate the utility of trained task embeddings for interpretation and visualization purposes.
Nivarthi, C. P. & Sick, B. (2023),
Towards Few-Shot Time Series Anomaly Detection with Temporal Attention and Dynamic Thresholding, in
'International Conference on Machine Learning and Applications (ICMLA)'
, IEEE,
, pp. 1444--1450
.
[BibTeX]
[Endnote]
Anomaly detection plays a pivotal role in diverse realworld applications such as cybersecurity, fault detection, network monitoring, predictive maintenance, and highly automated driving. However, obtaining labeled anomalous data can be a formidable challenge, especially when anomalies exhibit temporal evolution. This paper introduces LATAM (Long short-term memory Autoencoder with Temporal Attention Mechanism) for few-shot anomaly detection, with the aim of enhancing detection performance in scenarios with limited labeled anomaly data. LATAM effectively captures temporal dependencies and emphasizes significant patterns in multivariate time series data. In our investigation, we comprehensively evaluate LATAM against other anomaly detection models, particularly assessing its capability in few-shot learning scenarios where we have minimal examples from the normal class and none from the anomalous class in the training data. Our experimental results, derived from real-world photovoltaic inverter data, highlight LATAM’s superiority, showcasing a substantial 27% mean F1 score improvement, even when trained on a mere two-week dataset. Furthermore, LATAM demonstrates remarkable results on the open-source SWaT dataset, achieving a 12% boost in accuracy with only two days of training data. Moreover, we introduce a simple yet effective dynamic thresholding mechanism, further enhancing the anomaly detection capabilities of LATAM. This underscores LATAM’s efficacy in addressing the challenges posed by limited labeled anomalies in practical scenarios and it proves valuable for downstream tasks involving temporal representation and time series prediction, extending its utility beyond anomaly detection applications.
Schreck, S.; Reichert, H.; Hetzel, M.; Doll, K. & Sick, B. (2023),
Height Change Feature Based Free Space Detection, in
'International Conference on Control, Mechatronics and Automation (ICCMA)'
, IEEE,
, pp. 171--176
.
[BibTeX]
[Endnote]
We present a novel method for free space detection in dynamic environments like factory sites, crucial for autonomous forklifts to avoid collisions. It introduces a technique for fast surface normal estimation using spherical projected LiDAR data, which helps in detecting free space efficiently and in real-time. The method's effectiveness is proven with a 50.90% mIoU score on the Semantic KITTI dataset at 105 Hz, and a 63.30% mIoU score on a factory site dataset at 54 Hz.
Schreiber, J. & Sick, B. (2023),
'Model selection, adaptation, and combination for transfer learning in wind and photovoltaic power forecasts', Energy and AI
14
, 100249
.
[BibTeX]
[Endnote]
There is recent interest in using model hubs – a collection of pre-trained models – in computer vision tasks. To employ a model hub, we first select a source model and then adapt the model for the target to compensate for differences. There still needs to be more research on model selection and adaption for renewable power forecasts. In particular, none of the related work examines different model selection and adaptation strategies for neural network architectures. Also, none of the current studies investigates the influence of available training samples and considers seasonality in the evaluation. We close these gaps by conducting the first thorough experiment for model selection and adaptation for transfer learning in renewable power forecast, adopting recent developments from the field of computer vision on 667 wind and photovoltaic parks from six datasets. We simulate different amounts of training samples for each season to calculate informative forecast errors. We examine the marginal likelihood and forecast error for model selection for those amounts. Furthermore, we study four adaption strategies. As an extension of the current state of the art, we utilize a Bayesian linear regression for forecasting the response based on features extracted from a neural network. This approach outperforms the baseline with only seven days of training data and shows that fine-tuning is not beneficial with less than three months of data. We further show how combining multiple models through ensembles can significantly improve the model selection and adaptation approach such that we have a similar mean error with only 30 days of training data which is otherwise only possible with an entire year of training data. We achieve a mean error of 9.8 and 14 percent for the most realistic dataset for PV and wind with only seven days of training data.
Loeser, I.; Braun, M.; Gruhl, C.; Menke, J.-H.; Sick, B. & Tomforde, S. (2022),
'The Vision of Self-Management in Cognitive Organic Power Distribution Systems', Energies
15
(3)
, 881
.
[BibTeX]
[Endnote]
Due to the ongoing trend towards a decarbonisation of energy use, the power system is expected to become the backbone of all energy sectors and thus the fundamental critical infrastructure. High penetration with distributed energy resources demands the coordination of a large number of prosumers, partly controlled by home energy management systems (HEMS), to be designed in such a way that the power system’s operational limits are not violated. On the grid level, distribution management systems (DMS) seek to keep the power system in the normal operational state. On the prosumer level, distributed HEMS optimise the internal power flows by setpoint specification of batteries, photovoltaic generators, or flexible loads. The vision of the ODiS (Organic Distribution System) initiative is to develop an architecture to operate a distribution grid reliably, with high resiliency, and fully autonomously by developing “organic” HEMS and DMS which possess multiple self-x capabilities, collectively referred to as self-management. Thus, ODiS seeks answers to the following question: How can we create the most appropriate models, techniques, and algorithms to develop novel kinds of self-configuring, self-organising, self-healing, and self-optimising DMS that are integrally coupled with the distributed HEMS? In this concept paper, the vision of ODiS is presented in detail based on a thorough review of the state of the art.
Pham, T.; Kottke, D.; Krempl, G. & Sick, B. (2022),
'Stream-based active learning for sliding windows under the influence of verification latency', Machine Learning
111
(6)
, 2011--2036
.
[BibTeX]
[Endnote]
Stream-based active learning (AL) strategies minimize the labeling effort by querying labels that improve the classifier's performance the most. So far, these strategies neglect the fact that an oracle or expert requires time to provide a queried label. We show that existing AL methods deteriorate or even fail under the influence of such verification latency. The problem with these methods is that they estimate a label's utility on the currently available labeled data. However, when this label would arrive, some of the current data may have gotten outdated and new labels have arrived. In this article, we propose to simulate the available data at the time when the label would arrive. Therefore, our method Forgetting and Simulating (FS) forgets outdated information and simulates the delayed labels to get more realistic utility estimates. We assume to know the label's arrival date a priori and the classifier's training data to be bounded by a sliding window. Our extensive experiments show that FS improves stream-based AL strategies in settings with both, constant and variable verification latency.