2025 |
|
Characterization of explosives in a controlled blast chamber using single-particle mass spectrometer for post-blast particle analysis S. Jeong, J. Schade, L. Hettmanczyk, C. Ulrich, H. Ruser, F. Schnürer, M. Adelhardt and T. Adam; Talanta 298 (2025) 128986. |
| Abstract: The detection and characterization of post-blast residues is critical for forensic attribution, explosive safety assessments, and environmental monitoring. In this study, five high explosives, HMX, TNT, Composition B, HNS, and PETN, were detonated under controlled chamber conditions and analyzed in real-time using a single-particle mass spectrometer (SPMS). SPMS facilitates the chemical profiling of individual aerosol particles and can detect a wide range of chemical compounds without any sample preparation. For the first time, the incorporation of a focused laser beam during analysis makes it possible to investigate the spectral information of post-blast particles depending on the explosive. The resulting mass spectra showed clear compositional differences among the explosives: HMX, TNT, and Composition B produced strong polycyclic aromatic hydrocarbon signals, while HNS and PETN generated sparse ion profiles. Additionally, inorganic ions originating from stabilizers and primers were detected. These results demonstrate the capability of SPMS to distinguish explosive formulations based on their post-detonation aerosol signatures. The novel method supports rapid screening and detailed chemical classification, offering a powerful tool for real-time forensic investigations, explosive event reconstruction, and future data-driven identifications. |
BibTeX:
@article{Jeong2025,
author = {Seongho Jeong and Julian Schade and Lara Hettmanczyk and Christian Ulrich and Heinrich Ruser and Frank Schnürer and Mario Adelhardt and Thomas Adam},
title = {Characterization of explosives in a controlled blast chamber using single-particle mass spectrometer for post-blast particle analysis},
journal = {Talanta},
type = {OpenAccess},
school = {Universität der Bundeswehr München, Fakultät für Luft- und Raumfahrttechnik, LRT 2 - Institut für Angewandte Physik und Messtechnik, Professur: Dollinger, Günther},
year = {2025},
volume = {298},
pages = {128986},
url = {https://www.sciencedirect.com/science/article/pii/S0039914025014778},
doi = {https://doi.org/10.1016/j.talanta.2025.128986}
}
|
|
Single-Particle Mass Spectrometry coupled with Deep Learning Approaches for automatic on-site Classification of Aerosol Particles H. Ruser, G. Wang, J. Schade, J. Passig, R. Zimmermann and T. Adam; In: 2025 IEEE International Instrumentation and Measurement Technology Conference (I2MTC) (2025) 1-5 . |
| Abstract: Aerosol particle analysis has received greater attention recently, as many studies have linked it to the impacts on climate change, environmental pollution, human health and safety. However, due to the complex sources and evolution of the composition of aerosol particles, particulate matter control remains a challenge. With its high sensitivity and specificity, Single-Particle Mass Spectrometry (SPMS) is a powerful technology to identify the chemical signature of individual aerosol particles in real-time, detectable even after long-distance air transport from the source. Our new general method to evaluate different pollutant and hazardous particle emissions adaptively identifies unique and stable signatures in the mass spectra and classifies the particles on-site in real-time, based on a supervised deep-learning approach. Examples of applications range from particle classification from combustion processes up to the identification of hazardous substances from warfare agents, explosives, narcotics and drugs. |
BibTeX:
@inproceedings{Ruser2025,
author = {Ruser, Heinrich and Wang, Guanzhong and Schade, Julian and Passig, Johannes and Zimmermann, Ralf and Adam, Thomas},
title = {Single-Particle Mass Spectrometry coupled with Deep Learning Approaches for automatic on-site Classification of Aerosol Particles},
booktitle = {2025 IEEE International Instrumentation and Measurement Technology Conference (I2MTC)},
type = {OpenAccess},
school = {Universität der Bundeswehr München, Fakultät für Luft- und Raumfahrttechnik, LRT 2 - Institut für Angewandte Physik und Messtechnik, Professur: Dollinger, Günther},
year = {2025},
pages = {1-5},
url = {https://ieeexplore.ieee.org/document/11079167},
doi = {https://doi.org/10.1109/I2MTC62753.2025.11079167}
}
|
|
Deep learning based aerosol particle classification for the detection of ship emissions G. Wang, H. Ruser, J. Schade, S. Jeong, J. Passig, R. Zimmermann, G. Dollinger and T. Adam; Science of The Total Environment 994 (2025) 180041. |
| Abstract: Increasing recognition of the impact of shipping on air pollution has led the International Maritime Organization (IMO) to establish Sulfur Emission Control Areas (SECA) to reduce emissions. Within SECA, ships must switch to low-sulfur fuel or use a scrubber technique to clean their exhaust gases. Conventional monitoring methods are limited by detection range, real-time data availability, and challenges in source attribution. This study describes a monitoring system that combines single-particle mass spectrometry (SPMS) with deep learning to overcome these shortcomings. SPMS can reveal the chemical composition of individual airborne aerosol particles, with the capability to detect emissions over several kilometers, enabling real-time pollution source identification. To automatically process the complex mass spectral data, a convolutional neural network (CNN) was designed, achieving 92 % accuracy in classifying 13 distinct classes of abundant aerosol particles. The results demonstrate that the proposed detection system enables to automatically classify aerosol particles from multiple sources. Of particular concern in this study is the in-situ analysis of particles from ship exhaust plumes, to rapidly identify ships running on polluting heavy fuel oil. Focusing on unique particles containing vanadium (51V+/67[VO]+), nickel (58/60Ni+), and iron (54/56Fe+) ions, designated as V-rich class, the real-time classification makes it possible to reliably detect particles from heavy fuel oil (HFO) combustion. In addition, to locate the emission sources, the CNN's predictions are linked to local wind measurements and ship trajectories provided by the Automatic Identification System (AIS). During a one-week monitoring period, 21 ships passing the measurement site 80 times in distances of up to ∼1.3 km were detected using HFO. |
BibTeX:
@article{Wang2025,
author = {Wang, Guanzhong and Ruser, Heinrich and Schade, Julian and Jeong, Seongho and Passig, Johannes and Zimmermann, Ralf and Dollinger, Günther and Adam, Thomas},
title = {Deep learning based aerosol particle classification for the detection of ship emissions},
journal = {Science of The Total Environment},
publisher = {Elsevier BV},
type = {OpenAccess},
school = {Universität der Bundeswehr München, Fakultät für Luft- und Raumfahrttechnik, LRT 2 - Institut für Angewandte Physik und Messtechnik, Professur: Dollinger, Günther},
year = {2025},
volume = {994},
pages = {180041},
url = {https://www.sciencedirect.com/science/article/pii/S004896972501681X},
doi = {https://doi.org/10.1016/j.scitotenv.2025.180041}
}
|
|
Harmonizing Machine Learning and Numerical Dispersion Modelling for Improved Ship Emission Source Identification G. Wang, R. Badeke, H. Ruser, J. Schade and T. Adam; In: Proceedings of the 23rd International Conference onHarmonisation within Atmospheric Dispersion Modelling for Regulatory Purposes (2025) . |
| Abstract: Monitoring emissions from the fuel combustion of seagoing ships is a prerequisite for assessing their harmful impacts and for seamlessly checking whether local regulations are complied with. We present results from a measurement campaign in Darsser Ort on the south coast of the Baltic Sea, approx. 20 – 40 km east of the main shipping routes and approx. 40 km north-east of the port of Rostock (Germany), the largest freight and passenger port in the area. During the two-month-long campaign the chemical composition of about 1.6 million individual aerosol particles was analyzed using single-particle mass spectrometry (SPMS). We developed a deep learning model to automatically classify the SPMS data according to characteristic ion combinations in the bipolar mass spectra, in order to analyze ship fuel emissions in real time. Combined with local wind data (speed and direction) and Automatic Identification System (AIS) ship identification and position data, ships operating on polluting types of fuel could be identified and localized. We verified our findings on back-traced ship engine emissions by simulating exhaust plume distributions using a sophisticated aerosol transport model (EPISODE-CityChem). In many cases, the model chain successfully replicates emission events detected by chemical profiling of individual aerosol particles with matching temporal detail. In particular, correlating the results of both approaches leads to robust and reliable assignment of detected emissions to specific ships. |
BibTeX:
@inproceedings{Wang2025a,
author = {Wang, Guanzhong and Badeke, Ronny and Ruser, Heinrich and Schade, Julian and Adam, Thomas},
title = {Harmonizing Machine Learning and Numerical Dispersion Modelling for Improved Ship Emission Source Identification},
booktitle = {Proceedings of the 23rd International Conference onHarmonisation within Atmospheric Dispersion Modelling for Regulatory Purposes},
school = {Monitoring emissions from the fuel combustion of seagoing ships is a prerequisite for assessing their harmful impacts and for seamlessly checking whether local regulations are complied with. We present results from a measurement campaign in Darsser Ort on the south coast of the Baltic Sea, approx. 20 – 40 km east of the main shipping routes and approx. 40 km north-east of the port of Rostock (Germany), the largest freight and passenger port in the area. During the two-month-long campaign the chemical composition of about 1.6 million individual aerosol particles was analyzed using single-particle mass spectrometry (SPMS). We developed a deep learning model to automatically classify the SPMS data according to characteristic ion combinations in the bipolar mass spectra, in order to analyze ship fuel emissions in real time. Combined with local wind data (speed and direction) and Automatic Identification System (AIS) ship identification and position data, ships operating on polluting types of fuel could be identified and localized. We verified our findings on back-traced ship engine emissions by simulating exhaust plume distributions using a sophisticated aerosol transport model (EPISODE-CityChem). In many cases, the model chain successfully replicates emission events detected by chemical profiling of individual aerosol particles with matching temporal detail. In particular, correlating the results of both approaches leads to robust and reliable assignment of detected emissions to specific ships.�},
year = {2025},
url = {https://www.conferences.uni-hamburg.de/event/602/}
}
|
2024 |
|
Machine learning approaches for automatic classification of single-particle mass spectrometry data G. Wang, H. Ruser, J. Schade, J. Passig, T. Adam, G. Dollinger and R. Zimmermann; Atmospheric Measurement Techniques 17 (1) (2024) 299-313. |
| Abstract: The chemical composition of aerosol particles is a key parameter for human health and climate effects. Single-particle mass spectrometry (SPMS) has evolved to a mature technology with unique chemical coverage and the capability to analyze the distribution of aerosol components in the particle ensemble in real time. With the fully automated characterization of the chemical profile of the aerosol particles, selective real-time monitoring of air quality could be performed, e.g., for urgent risk assessments due to particularly harmful pollutants. For aerosol particle classification, mostly unsupervised clustering algorithms (ART-2a, K-means and their derivatives) are used, which require manual postprocessing. In this work, we focus on supervised algorithms to tackle the problem of the automatic classification of large amounts of aerosol particle data. Supervised learning requires data with labels to train a predictive model. Therefore, we created a labeled benchmark dataset containing ∼ 24 000 particles with eight different coarse categories that were highly abundant at a measurement in summer in Central Europe: elemental carbon (EC), organic carbon and elemental carbon (OC-EC), potassium-rich (K-rich), calcium-rich (Ca-rich), iron-rich (Fe-rich), vanadium-rich (V-rich), magnesium-rich (Mg-rich) and sodium-rich (Na-rich). Using the chemical features of particles, the performance of the following classical supervised algorithms was tested: K-nearest neighbors, support vector machine, decision tree, random forest and multi-layer perceptron. This work shows that despite the entrenched position of unsupervised clustering algorithms in the field, the use of supervised algorithms has the potential to replace the manual step of clustering algorithms in many applications, where real-time data analysis is essential. For the classification of the eight classes, the prediction accuracy of several supervised algorithms exceeded 97 %. The trained model was used to classify ∼ 49 000 particles from a blind dataset in 0.2 s, taking into account also a class of “unclassified” particles. The predictions are highly consistent with the results obtained in a previous study using ART-2a. |
BibTeX:
@article{Wang2024,
author = {Wang, G. and Ruser, H. and Schade, J. and Passig, J. and Adam, T. and Dollinger, G. and Zimmermann, R.},
title = {Machine learning approaches for automatic classification of single-particle mass spectrometry data},
journal = {Atmospheric Measurement Techniques},
type = {OpenAccess},
school = {Universität der Bundeswehr München, Fakultät für Luft- und Raumfahrttechnik, LRT 2 - Institut für Angewandte Physik und Messtechnik, Professur: Dollinger, Günther},
year = {2024},
volume = {17},
number = {1},
pages = {299--313},
url = {https://amt.copernicus.org/articles/17/299/2024/},
doi = {https://doi.org/10.5194/amt-17-299-2024}
}
|
|
A Fuzzy Convolutional Neural Network for the Classification of Aerosol Particle Mass Spectral Patterns Generated by Single-Particle Mass Spectrometry G. Wang, H. Ruser, J. Schade, J. Passig, R. Zimmermann, G. Dollinger and T. Adam; In: 2024 International Joint Conference on Neural Networks (IJCNN) (2024) 1-8 . |
| Abstract: Air quality control is essential for assessing the impact on human health, environment and climate. Single-particle mass spectrometry (SPMS) is a powerful measurement tool for providing the chemical composition of air-transported particle matter (PM) in real-time. Common methods to classify PM according to characteristic ion patterns in their mass spectra are based on clustering methods which generally require manual postprocessing and are not suitable for real-time automated air quality monitoring. A number of automated classification models trained on labeled SPMS data were proposed recently by the authors. As it appeared, the most advanced methods of them, based on deep-learning convolutional neural networks (CNN), still have difficulties in distinguishing between classes of similar but distinctive mass spectra. In this work, we propose a novel fuzzy convolutional neural network (FCNN) combining fuzzy network and CNN to accurately classify particle mass spectral patterns. FCNN models integrate the respective advantages of fuzzy and neural networks to effectively separate non-isolated and overlapping features of similar patterns through fuzzy information, and also inherently optimize the fuzzy rule parameters through NN back propagation. To validate the performance of FCNN, a benchmark dataset with 37,406 samples in 13 particle classes was created. Compared to CNN, with FCNN 10 out of 13 classes could be classified with higher accuracy, especially those distinguishing subtle differences in the mass spectra. Applied to automated SPMS analysis, the proposed FCNN tackles the classification challenges posed by the chemical complexity of aerosol particles and opens up ways to foster the development of specific, real-time air quality monitoring and pollution source identification systems. |
BibTeX:
@inproceedings{Wang2024a,
author = {Wang, Guanzhong and Ruser, Heinrich and Schade, Julian and Passig, Johannes and Zimmermann, Ralf and Dollinger, Günther and Adam, Thomas},
title = {A Fuzzy Convolutional Neural Network for the Classification of Aerosol Particle Mass Spectral Patterns Generated by Single-Particle Mass Spectrometry},
booktitle = {2024 International Joint Conference on Neural Networks (IJCNN)},
type = {OpenAccess},
school = {Universität der Bundeswehr München, Fakultät für Luft- und Raumfahrttechnik, LRT 2 - Institut für Angewandte Physik und Messtechnik, Professur: Dollinger, Günther},
year = {2024},
pages = {1-8},
url = {https://ieeexplore.ieee.org/document/10650883},
doi = {https://doi.org/10.1109/IJCNN60899.2024.10650883}
}
|
|
Rapid Classification Of Aerosol Particle Mass Spectra Using Data Augmentation And Deep Learning G. Wang, H. Ruser, J. Schade, J. Passig, R. Zimmermann, G. Dollinger and T. Adam; In: 2024 IEEE Conference on Artificial Intelligence (CAI) (2024) 1167-1172 . |
| Abstract: The concentration and chemical composition of airborne aerosol particles are important indicators of air quality and sources of air pollution. The particles’ chemical composition reveals probable emission sources, like traffic, biomass burning, wildfires, agriculture, or industrial sources. Single-particle mass spectrometry (SPMS), combined with rapid spectral classification, uniquely enables an in-situ analysis of the chemical composition of individual aerosol particles in real-time for environmental monitoring and other tasks. Modern SPMS devices analyze hundreds of individual particles per minute. Rapid and accurate classification of such large amounts of data remains challenging. Conventional clustering algorithms require tedious manual post-processing. A mass spectrum can be understood as a 1D image per analyzed particle. We applied CNN-based algorithms to perform a fully automated classification. To train the models, usually a large amount of labeled data needs to be prepared. With a manually created benchmark dataset containing 10,400 samples in 13 classes of emission sources (800 samples per class) we achieved an accuracy of 90%. If the models are trained using only 100 labeled samples per class (1/8 labeled data), the models’ accuracy drops significantly to 75%. We explored suitable augmentation methods to improve the reliability and performance of multi-class classification for aerosol particle mass spectra in case of limited labeled data (1/8 labeled data). The results using the augmented data improved from 75% to 86.8%. This paves the way to sharply reduce the expensive and time-consuming work of expert labeling. Furthermore, we verified that converting the 1D mass spectrum into 2D representations and classifying them using 2D-CNN is more efficient than 1D-CNN networks, whether with or without data augmentation. |
BibTeX:
@inproceedings{Wang2024b,
author = {Wang, Guanzhong and Ruser, Heinrich and Schade, Julian and Passig, Johannes and Zimmermann, Ralf and Dollinger, Günther and Adam, Thomas},
title = {Rapid Classification Of Aerosol Particle Mass Spectra Using Data Augmentation And Deep Learning},
booktitle = {2024 IEEE Conference on Artificial Intelligence (CAI)},
type = {OpenAccess},
school = {Universität der Bundeswehr München, Fakultät für Luft- und Raumfahrttechnik, LRT 2 - Institut für Angewandte Physik und Messtechnik, Professur: Dollinger, Günther},
year = {2024},
pages = {1167-1172},
url = {https://ieeexplore.ieee.org/document/10605430},
doi = {https://doi.org/10.1109/CAI59869.2024.00208}
}
|
|
CNN-Based Aerosol Particle Classification Using 2D Representations of Single-Particle Mass Spectrometer Data G. Wang, H. Ruser, J. Schade, J. Passig, R. Zimmermann, G. Dollinger and T. Adam; In: 2024 International Conference on Artificial Intelligence in Information and Communication (ICAIIC) (2024) 205-210 . |
| Abstract: Single-particle mass spectrometry (SPMS) is a powerful real-time measurement technique to analyze the chemical composition of atmospheric aerosol particles: individual particles are desorbed and ionized to generate a bipolar mass spectrum that expresses the particle's chemical composition, giving clues to its origin and atmospheric processes. Popular approaches to classify SPMS data rely on clustering algorithms, resulting in the inability to achieve automated classification. Here, we present a modified deep learning approach for automatic classification of SPMS data in real-time. Before being processed by a convolutional neural network (CNN), the one-dimensional (1D) mass spectrum is converted into a two-dimensional (2D) representation, since in 2D, global and local features of the spectra are extracted more efficiently. Trained on real-world aerosol mass spectra from a month-long field measurement campaign, the proposed 2D-CNN model achieves a high mean classification accuracy of 92%, outperforming several well-known algorithms based on 2D-CNN, as well as a recently proposed 1D-CNN algorithm trained using 1D representations of mass spectra. |
BibTeX:
@inproceedings{Wang2024c,
author = {Wang, Guanzhong and Ruser, Heinrich and Schade, Julian and Passig, Johannes and Zimmermann, Ralf and Dollinger, Günther and Adam, Thomas},
title = {CNN-Based Aerosol Particle Classification Using 2D Representations of Single-Particle Mass Spectrometer Data},
booktitle = {2024 International Conference on Artificial Intelligence in Information and Communication (ICAIIC)},
type = {OpenAccess},
school = {Universität der Bundeswehr München, Fakultät für Luft- und Raumfahrttechnik, LRT 2 - Institut für Angewandte Physik und Messtechnik, Professur: Dollinger, Günther},
year = {2024},
pages = {205-210},
url = {https://ieeexplore.ieee.org/document/10463253},
doi = {https://doi.org/10.1109/ICAIIC60209.2024.10463253}
}
|
2023 |
|
1D-CNN Network Based Real-Time Aerosol Particle Classification With Single-Particle Mass Spectrometry G. Wang, H. Ruser, J. Schade, J. Passig, T. Adam, G. Dollinger and R. Zimmermann; IEEE Sensors Letters 7 (11) (2023) 1-4. |
| Abstract: Single-particle mass spectrometry (SPMS) is a measurement technique that aims to identify the chemical composition of individual airborne aerosol particles (PM 1 or PM 2.5) in real time. One-dimensional (1-D) spectral data of aerosol particles generated by SPMS carry rich information about the chemical composition associated with the sources of the particles, e.g., traffic and ship emissions, biomass burning, etc. Accurate classification of aerosol particles is essential to understand their sources and effects on human health. This letter investigates the application of SPMS and 1-D-convolutional neural network (1D-CNN) in aerosol particle classification. The proposed 1D-CNN achieved a mean classification accuracy of 90.4% with 13 particle classes. According to the experimental results, the combination of SPMS and 1D-CNN enables real-time collection, analysis, and classification of airborne aerosol particles to be used for highly responsive automated air quality monitoring. |
BibTeX:
@article{Wang2023,
author = {Wang, Guanzhong and Ruser, Heinrich and Schade, Julian and Passig, Johannes and Adam, Thomas and Dollinger, Günther and Zimmermann, Ralf},
title = {1D-CNN Network Based Real-Time Aerosol Particle Classification With Single-Particle Mass Spectrometry},
journal = {IEEE Sensors Letters},
type = {OpenAccess},
school = {Universität der Bundeswehr München, Fakultät für Luft- und Raumfahrttechnik, LRT 2 - Institut für Angewandte Physik und Messtechnik, Professur: Dollinger, Günther},
year = {2023},
volume = {7},
number = {11},
pages = {1-4},
url = {https://ieeexplore.ieee.org/document/10251644},
doi = {https://doi.org/10.1109/LSENS.2023.3315554}
}
|
2021 |
|
Verfahren zur Bestimmung des Verschmutzungsgrades von Thermoelementen im Betrieb D. Felkl and H. Ruser; In: 15. Dresdner Sensor-Symposium 2021 , Dresdner Sensor-Symposium (2021) 332-337 , AMA Association for Sensors and Measurement. |
| Abstract: Für die Verbesserung der Zuverlässigkeit eines Sensors und die Minimierung der Messabweichungen spielt die Kenntnis des Sensorzustandes eine wichtige Rolle. Dies gilt verstärkt im Zusammenhang mit der Sensorentwicklung für den Einsatz in komplexen industriellen Produktionsanlagen mit einer Vielzahl vernetzter sensorbasierter Teilsysteme (sog. cyber-physischer Systeme) [1]. Kontinuierliche Zustandsüberwachungen der Sensoren sind die Voraussetzung für eine prädiktive Wartung bzw. Instandsetzung. |
BibTeX:
@inproceedings{Felkl2021,
author = {Felkl, Dieter and Ruser, Heinrich},
title = {Verfahren zur Bestimmung des Verschmutzungsgrades von Thermoelementen im Betrieb},
booktitle = {15. Dresdner Sensor-Symposium 2021},
journal = {Dresdner Sensor-Symposium},
publisher = {AMA Association for Sensors and Measurement},
type = {OpenAccess},
school = {Universität der Bundeswehr München, Fakultät für Luft- und Raumfahrttechnik, LRT 2 - Institut für Angewandte Physik und Messtechnik, Professur: Dollinger, Günther},
year = {2021},
pages = {332--337},
url = {https://www.ama-science.org/proceedings/details/4134},
doi = {https://doi.org/10.5162/15dss2021/P13.3}
}
|
|
Phone-Pointing Remote App: Using Smartphones as Pointers in Gesture-Based IoT Remote Controls I. Kirsh and H. Ruser; In: , C. Stephanidis, M. Antona and S. Ntoa (Eds.), HCI International 2021 - Posters 1420 (2021) 14-21 , Springer International Publishing. |
| Abstract: Remote control mobile applications for operating Internet of Things (IoT) devices using smartphones are commonly based on a touch user interface. The effort of using such apps is often disproportionate to the simplicity of carrying out the actions manually. For example, turning a light on or off via menus and forms of a standard remote app might not be very convenient. A voice user interface, while easier to use, gives rise to other issues, including user privacy and distracting others nearby. This paper proposes a new type of universal IoT remote control applications for smartphones: phone-pointing remote apps. Using a phone-pointing remote app, users can physically point their smartphones at IoT devices to select them, and operate them via movement gestures, without needing to turn on the phone screen or talk, and with no need for any additional hardware. This new approach provides a unique combination of advantages. It is simple, intuitive, fast, and voiceless. Instead of using the touchscreen or the microphone as the input source, phone-pointing remote apps will use a combination of standard smartphone sensors, including the GNSS sensor, Wi-Fi scanner, Bluetooth receiver, camera, barometer, magnetometer, g-force meter, accelerometer, and gyroscope. An analysis of the proposed model in light of relevant results from related studies provides positive preliminary indications regarding the feasibility of this novel approach. |
BibTeX:
@inproceedings{Kirsh2021,
author = {Kirsh, Ilan and Ruser, Heinrich},
title = {Phone-Pointing Remote App: Using Smartphones as Pointers in Gesture-Based IoT Remote Controls},
booktitle = {HCI International 2021 - Posters},
publisher = {Springer International Publishing},
type = {OpenAccess},
school = {Universität der Bundeswehr München, Fakultät für Luft- und Raumfahrttechnik, LRT 2 - Institut für Angewandte Physik und Messtechnik, Professur: Dollinger, Günther},
year = {2021},
volume = {1420},
pages = {14--21},
editor = {Stephanidis, Constantine and Antona, Margherita and Ntoa, Stavroula},
url = {https://link.springer.com/chapter/10.1007/978-3-030-78642-7_3},
doi = {https://doi.org/10.1007/978-3-030-78642-7_3}
}
|
|
Evaluating the Accuracy and User Experience of a Gesture-Based Infrared Remote Control in Smart Homes H. Ruser, S. Vorwerg, C. Eicher, F. Pfeifer, F. Piela, A. Kaltenbach and L. Mechold; In: , M. Kurosu (Ed.), Human-Computer Interaction. Interaction Techniques and Novel Applications 12763 (2021) 89-108 , Springer International Publishing. |
| Abstract: To enhance user experience while satisfying basic expectations and needs is the most important goal in the design of assistive technical devices. As a contribution, the user experience with the SmartPointer, a novel hand-held gesture-based remote control for everyday use in the living environment, is being explored in comprehensive user tests. The concept and design of the SmartPointer exploits the user's familiarity with TV remotes, flashlights or laser pointers. The buttonless device emits both an infrared (IR) and a visible (VIS) laser beam and is designed to be universally and consistently used for a large variety of devices and appliances in private homes out of arm's reach. In the paper, the results of three user studies regarding recognition rates and usability issues are summarized. Study One was a mixed-method study in the pre-implementation stage with 20 older adults, gathering the expectations towards a gesture-based remote control and exploring simple, quasi-intuitive controlling gestures. In Study Two, the acceptance and usability of a prototype of the SmartPointer remote control was verified and compared with a group of 29 users from the target group, exploring 8 most frequently used gestures from Study One. In Study Three, comprehensive gesture-recognition tests with an updated version of the remote were carried out with a group of 11 younger adults in various light conditions, postures and distances to the operated device. All three studies confirm the feasibility of the underlying principle, the usability and satisfaction among the participants and the robustness of the technical solution along with a high success rate of the recognition algorithm. |
BibTeX:
@inproceedings{Ruser2021,
author = {Ruser, Heinrich and Vorwerg, Susan and Eicher, Cornelia and Pfeifer, Felix and Piela, Felix and Kaltenbach, André and Mechold, Lars},
title = {Evaluating the Accuracy and User Experience of a Gesture-Based Infrared Remote Control in Smart Homes},
booktitle = {Human-Computer Interaction. Interaction Techniques and Novel Applications},
publisher = {Springer International Publishing},
type = {OpenAccess},
school = {Universität der Bundeswehr München, Fakultät für Luft- und Raumfahrttechnik, LRT 2 - Institut für Angewandte Physik und Messtechnik, Professur: Dollinger, Günther},
year = {2021},
volume = {12763},
pages = {89--108},
editor = {Kurosu, Masaaki},
url = {https://link.springer.com/chapter/10.1007/978-3-030-78465-2_8},
doi = {https://doi.org/10.1007/978-3-030-78465-2_8}
}
|
|
''Point at It with Your Smartphone'': Assessing the Applicability of Orientation Sensing of Smartphones to Operate IoT Devices H. Ruser and I. Kirsh; In: , C. Stephanidis, M. Kurosu, J.Y.C. Chen, G. Fragomeni, N. Streitz, S. Konomi, H. Degen and S. Ntoa (Eds.), HCI International 2021 - Late Breaking Papers: Multimodality, eXtended Reality, and Artificial Intelligence 13095 (2021) 115-131 , Springer International Publishing. |
| Abstract: The built-in orientation and motion sensors of smartphones along with their wireless communication abilities are utilized to control connected IoT devices from any place in a room, by pointing at them with the smartphone in the hand. The information of which device is targeted will be derived from the user's actual location, the spatial orientation of the smartphone and pre-knowledge regarding the positions of devices. Chosen devices are remotely operated with simple mid-air gestures performed with the smartphone. The feasibility of this cost-effective approach is assessed by user experiments. The continuous readings of the smartphone's inclination, rotation and magnetic field sensors are recorded with a dedicated freeware app. An algorithm combines the sensor readings to deliver the actual spatial orientation. Our preliminary experiments with different smartphone models and several users show that pointing at defined positions and performing gestures with a smartphone in the user's hand can be accurately sensed without latency and with small deviations of the orientation measurements in the range of up to 5 degrees, indicating the feasibility of this novel approach. |
BibTeX:
@inproceedings{Ruser2021a,
author = {Ruser, Heinrich and Kirsh, Ilan},
title = {''Point at It with Your Smartphone'': Assessing the Applicability of Orientation Sensing of Smartphones to Operate IoT Devices},
booktitle = {HCI International 2021 - Late Breaking Papers: Multimodality, eXtended Reality, and Artificial Intelligence},
publisher = {Springer International Publishing},
type = {OpenAccess},
school = {Universität der Bundeswehr München, Fakultät für Luft- und Raumfahrttechnik, LRT 2 - Institut für Angewandte Physik und Messtechnik, Professur: Dollinger, Günther},
year = {2021},
volume = {13095},
pages = {115--131},
editor = {Stephanidis, Constantine and Kurosu, Masaaki and Chen, Jessie Y. C. and Fragomeni, Gino and Streitz, Norbert and Konomi, Shin'ichi and Degen, Helmut and Ntoa, Stavroula},
url = {https://link.springer.com/chapter/10.1007/978-3-030-90963-5_10},
doi = {https://doi.org/10.1007/978-3-030-90963-5_10}
}
|
2020 |
|
Making the Home Accessible - Experiments with an Infrared Handheld Gesture-Based Remote Control H. Ruser, S. Vorwerg and C. Eicher; In: , C. Stephanidis and M. Antona (Eds.), HCI International 2020 - Posters , Proceedings 22nd Int. Conf. Human-Computer InteractionInternational (HCII’20), Part III, 1226 (2020) 89-97 , Springer International Publishing. |
| Abstract: A universal remote control for many different technical devices in the living environment - which should be very easy, almost intuitively to use - would be most desirable, especially for elderly and mobility-impaired persons. For this purpose, a flashlight-like handheld infrared gesture-controlled remote we call SmartPointer is being developed and evaluated. Carrying out a mid-air gesture with the SmartPointer moves the irradiated beam of structured IR light across a receiver box near the device, which detects the light pattern's trajectory and converts the identified gesture into device-specific commands. In our laboratory study, the user experience of the SmartPointer system and its gesture recognition capabilities were examined with a group of 29 elderly volunteers who were asked to repeatedly carry out quasi-intuitive gestures with the SmartPointer to operate four typical home appliances: light switch (on/off), heating device (warmer/colder), blinds (up/down) and door (lock/unlock). Applying adaptive rule-based signal processing and pattern recognition, an overall recognition rate of 94.3% could be achieved so far. |
BibTeX:
@inproceedings{Ruser2020,
author = {Ruser, Heinrich and Vorwerg, Susan and Eicher, Cornelia},
title = {Making the Home Accessible - Experiments with an Infrared Handheld Gesture-Based Remote Control},
booktitle = {HCI International 2020 - Posters},
journal = {Proceedings 22nd Int. Conf. Human-Computer InteractionInternational (HCII’20), Part III,},
publisher = {Springer International Publishing},
school = {Universität der Bundeswehr München, Fakultät für Luft- und Raumfahrttechnik, LRT 2 - Institut für Angewandte Physik und Messtechnik, Professur: Dollinger, Günther},
year = {2020},
volume = {1226},
pages = {89--97},
editor = {Stephanidis, Constantine and Antona, Margherita},
url = {https://link.springer.com/chapter/10.1007/978-3-030-50732-9_13},
doi = {https://doi.org/10.1007/978-3-030-50732-9_13}
}
|
|
Low-cost gestural interaction based on motion estimation of a projected dot pattern : Experiments with a working prototype H. Ruser, A. Kaltenbach, L. Mechold, F. Obée and F. Piela; In: 2020 IEEE SENSORS , Proceedings of IEEE Sensors 2020-October (2020) 1-4 , Institute of Electrical and Electronics Engineers (IEEE). |
| Abstract: The concept and realization of an easy-to-use, low-cost handheld IR optical sensor for universal, long-range gesture-controlled operation of technical devices is presented and recognition results with a working prototype are given. The system comprises two elements: a handheld buttonless light emitter which resembles a small laser pointer but emits a wide-angle IR random-dot pattern along with a visible light beam (for device selection) and a small photo-sensor array along with a processing/decoder unit near or in the device to be controlled. The emitted IR light is spatially modulated by a diffractive optical element (DOE) which has a specially designed micro-structured surface to generate a binary pseudo-random array of light beams. This concept allows to come up with a handheld device which contains only a few inexpensive mass-market components (since the IR-optical signal is both the information and transmission channel). User experience and gesture recognition capabilities were examined with a group of 29 elderly volunteers who were asked to repeatedly "draw" quasi-intuitive gestures with the handheld emitter towards typical home appliances like light switch, blinds, heating device, and door from distances of up to 6 meters. Applying a correlation-based reconstruction algorithm and adaptive rule-based pattern recognition with low computational load for real-time processing on low-end processors, an overall recognition rate of 94 % could be achieved so far. As the algorithm is being optimized to enable sequential spotting of meaningful gestures and enhanced class separation, the list of gestures will be extended. |
BibTeX:
@inproceedings{Ruser2020a,
author = {H. Ruser and A. Kaltenbach and L. Mechold and F. Obée and F. Piela},
title = {Low-cost gestural interaction based on motion estimation of a projected dot pattern : Experiments with a working prototype},
booktitle = {2020 IEEE SENSORS},
journal = {Proceedings of IEEE Sensors},
publisher = {Institute of Electrical and Electronics Engineers (IEEE)},
school = {Universität der Bundeswehr München, Fakultät für Luft- und Raumfahrttechnik, LRT 2 - Institut für Angewandte Physik und Messtechnik, Professur: Dollinger, Günther},
year = {2020},
volume = {2020-October},
number = {9278920},
pages = {1--4},
url = {https://ieeexplore.ieee.org/document/9278920},
doi = {https://doi.org/10.1109/SENSORS47125.2020.9278920}
}
|
|
Vergleichsstudie zur Akzeptanz und Usability eines gestengesteuerten Smart Home Systems mit Personen im späten Erwachsenenalter (Comparative study on acceptance and usability of agesture-controlled smart home system with persons in late adulthood) S. Vorwerg, C. Eicher, H. Ruser, F. Piela, FelixObee and L. Mechold; In: 54th Annual Conference of the GermanSociety for Biomedical Engineering (BMT 2020) , Proc. AAL-Kongress 2020 (2020) 47-52 , VDE-Verlag. |
| Abstract: Kurzfassung Hintergrund: Im Rahmen des Forschungs- und Entwicklungsprojektes „SmartPointer“ soll für die Gruppe älterer Men-schen eine tastenlose gestengesteuerte Fernbedienung mit einer einfachen, quasi-intuitiven Bedienstruktur entwickelt und evaluiert werden. Ziel: Um den vorherrschenden Akzeptanzproblemen gegenüber Smart-Home-Technologien im Alter entgegenzuwirken, war das Ziel der hier präsentierten Studie die Untersuchung der Technikbereitschaft, Akzeptanz und Usability bei der Bedienung von vier Smart-Home-Demonstratoren durch die Zielgruppe. Methodik: Für die Studie wurden 29 gesunde Probanden ab 60 Jahren in zwei Altersgruppen (unter und über 75 Jahren) aufgeteilt. In Form eines Wizard-of-Oz Experiments konnte jede Person die Anwendungsszenarien für Licht, Rollo, Hei-zung sowie Tür mit einer von zwei Fernbedienungen erproben. Jede Anwendung erforderte eine spezifische Geste. Die Erhebung der Daten erfolgte mit verschiedenen Assessments sowie einem Kurzinterview. Ergebnisse: Keine Unterschiede ergaben sich bei der Technikbereitschaft zwischen beiden Altersgruppen. Signifikante Unterschiede bestanden in der Wahl der Fernbedienung (Chi2-Test, p=0,008). Ein Gruppenunterschied wurde auch in der Bewertung der Usability deutlich (unabh. t-Test, p=0,035). Insgesamt wies das System eine Gestenerkennungsrate von etwa 95% auf. Für die Nutzerakzeptanz spielen Features, wie bspw. ein optisches Feedback eine Rolle. Die Akzeptanz und Usability kann zudem durch eine bessere Oberflächengestaltung der Fernbedienung und Verbesserung der Reakti-onszeit des Systems erhöht werden. Schlussfolgerung: Insgesamt wurden die evaluierten Einsatzszenarien von den Probanden als alltagstauglich bewertet. Als Anwendungsfälle wurden explizit Bettlägerigkeit oder Immobilität genannt, um damit die Selbstständigkeit zu erhal-ten. Um die Funktionalität und somit Usability sowie Nutzerakzeptanz zu steigern, werden alle Systemkomponenten im weiteren Projektverlauf verbessert und erneut evaluiert. Abstract |
BibTeX:
@inproceedings{Vorwerg2020,
author = {Vorwerg, Susan and Eicher, Cornelia and Ruser, Heinrich and Piela, Felix and FelixObee and Mechold, Lars},
title = {Vergleichsstudie zur Akzeptanz und Usability eines gestengesteuerten Smart Home Systems mit Personen im späten Erwachsenenalter (Comparative study on acceptance and usability of agesture-controlled smart home system with persons in late adulthood)},
booktitle = {54th Annual Conference of the GermanSociety for Biomedical Engineering (BMT 2020)},
journal = {Proc. AAL-Kongress 2020},
publisher = {VDE-Verlag},
school = {Universität der Bundeswehr München, Fakultät für Luft- und Raumfahrttechnik, LRT 2 - Institut für Angewandte Physik und Messtechnik, Professur: Dollinger, Günther},
year = {2020},
pages = {47--52},
url = {https://www.vde-verlag.de/buecher/455342/aal-kongress-2020.html}
}
|
2019 |
|
”SmartPointer“- Buttonless remote control based on structured light and intuitive gestures H. Ruser, A. Kaltenbach and L. Mechold; In: , M.A. Thomas Kirste (Ed.), Proceedings of the 6th international Workshop on Sensor-based Activity Recognition and Interaction ( iWOAR 2019) , ACM International Conference Proceeding Series (2019) 1-6 . |
| Abstract: The concept and design guidelines for an universal gesture-based remote control with simple intuitive operating are presented. The hand-held buttonless “SmartPointer” emits “structured” infrared (IR) light with a spatial pattern projected by a diffractive optical element (DOE). A cost-effective array of photodiodes on or near the devices to be remotely controlled (such as light, blinds, windows, heating/air conditioning or TV/radio) records the light intensities while a gesture is carried out. Based on the cross-correlation of the spatio-temporal intensity changes at all pairs of photodiodes, the trajectory of motion of the pattern is reconstructed and the gesture is recognized (classified). During tests with uninstructed persons, horizontal, vertical, circular and targeting gestures were most frequently used. Extensive simulations addressing design parameters of the 2D projection pattern and the receiver array were carried out to identify suitable parameter sets for highly reliable gesture reconstruction and recognition of the recorded gestures. As a result, DOE with various pseudo-random beam distributions can be applied. |
BibTeX:
@inproceedings{Ruser2019,
author = {Ruser, H. and Kaltenbach, A. and Mechold, L.},
title = {”SmartPointer“- Buttonless remote control based on structured light and intuitive gestures},
booktitle = {Proceedings of the 6th international Workshop on Sensor-based Activity Recognition and Interaction ( iWOAR 2019)},
journal = {ACM International Conference Proceeding Series},
school = {Universität der Bundeswehr München, Fakultät für Luft- und Raumfahrttechnik, LRT 2 - Institut für Angewandte Physik und Messtechnik, Professur: Dollinger, Günther},
year = {2019},
number = {3361691},
pages = {1--6},
editor = {Thomas Kirste, Mario Aehnelt},
url = {https://dl.acm.org/doi/10.1145/3361684.3361691},
doi = {https://doi.org/10.1145/3361684.3361691}
}
|
|
Optical sensor based on pseudo-random diffractive optical elements for reliable gesture reconstruction H. Ruser, A. Kaltenbach, L. Mechold, F. Obee and F. Piela; In: 2019 IEEE SENSORS , Proceedings of IEEE Sensors 2019-October (2019) 1-4 , IEEE. |
| Abstract: The concept, design guidelines and reconstruction results for a universal gesture-based optical remote control with simple quasi-intuitive operation are presented. The buttonless hand-held flashlight-type device emits structured infrared light with a pseudo-random spatial pattern projected by a diffractive optical element (DOE). A cost-effective array of photodetectors on or near the device to be remotely controlled records the spatio-temporal intensity changes while a gesture is carried out. From the consecutive time lags between highly correlated signal segments received at each pair of photodetectors, the velocity vector is composed from which Cartesian coordinates of the trajectory of motion of the pattern are calculated and the gesture is reconstructed. Extensive simulations varying major design parameters of the DOE pattern and the receiver array were carried out. Based on simulated and typical practical gestures obtained from user tests, design parameters for a highly satisfactory reconstruction performance could be identified. |
BibTeX:
@inproceedings{Ruser2019a,
author = {Ruser, H. and Kaltenbach, A. and Mechold, L. and Obee, F. and Piela, F.},
title = {Optical sensor based on pseudo-random diffractive optical elements for reliable gesture reconstruction},
booktitle = {2019 IEEE SENSORS},
journal = {Proceedings of IEEE Sensors},
publisher = {IEEE},
school = {Universität der Bundeswehr München, Fakultät für Luft- und Raumfahrttechnik, LRT 2 - Institut für Angewandte Physik und Messtechnik, Professur: Dollinger, Günther},
year = {2019},
volume = {2019-October},
number = {8956875},
pages = {1--4},
url = {https://ieeexplore.ieee.org/document/8956875},
doi = {https://doi.org/10.1109/SENSORS43011.2019.8956875}
}
|
|
Entwurf einer universellen tastenlosen optischen Fernbedienung H. Ruser; In: 14. Dresdner Sensor-Symposium 2019 , Dresdner Sensor-Symposium (2019) 182-186 , AMA Association for Sensors and Measurement. |
| Abstract: Im Beitrag wird der Entwurf eines optischen Systems zur universellen Fernbedienung vieler technischer Geräte und Systeme im Haushalt, in Gebäuden oder in Industrie- Umgebungen beschrieben: Mittels eines leichten und tastenlosen „Lichtzeigers“, der wie eine kleine Taschenlampe in der Hand liegt, werden quasi-intuitive Zeige- und Bewegungsgesten in Richtung des zu bedienenden Gerätes ausgeführt, um dieses auszuwählen, zu schalten oder dessen Einstellungen in Stufen oder stufenlos zu verändern. Diese natürliche Mensch-Maschine-Schnittstelle soll insbesondere dazu beitragen, Personen mit eingeschränkter Mobilität und älteren Menschen ein längeres selbständiges Leben in den eigenen „vier Wänden“ zu erleichtern. Aber auch für andere Nutzergruppen dürfte die einfache Bedienbarkeit und die universelle Einsetzbarkeit des „Lichtzeigers“ ein Zugewinn an Komfort und Bequemlichkeit bedeuten. Der Verzicht auf zu erlernende Gesten und der ausgesprochen geringe Hardware-Aufwand stellen dabei wesentliche Unterscheidungen zu alternativen Ansätzen dar. |
BibTeX:
@inproceedings{Ruser2019b,
author = {Heinrich Ruser},
title = {Entwurf einer universellen tastenlosen optischen Fernbedienung},
booktitle = {14. Dresdner Sensor-Symposium 2019},
journal = {Dresdner Sensor-Symposium},
publisher = {AMA Association for Sensors and Measurement},
type = {Open Access},
school = {Universität der Bundeswehr München, Fakultät für Luft- und Raumfahrttechnik, LRT 2 - Institut für Angewandte Physik und Messtechnik, Professur: Dollinger, Günther},
year = {2019},
pages = {182--186},
url = {https://www.ama-science.org/proceedings/details/3546},
doi = {https://doi.org/10.5162/14dss2019/P2.05}
}
|
|
Requirements for Gesture-Controlled Remote Operation to Facilitate Human-Technology Interaction in the Living Environment of Elderly People S. Vorwerg, C. Eicher, H. Ruser, F. Piela, F. Obée, A. Kaltenbach and L. Mechold; In: , S.G. Zhou J. (Ed.), Human Aspects of IT for the Aged Population. Design for the Elderly and Technology Acceptance. HCII 2019. , Lecture Notes in Computer Science 11592 (2019) 551-569 , Springer. |
| Abstract: The “SmartPointer” (SP) technology comprises a universal buttonless gesture-controlled handheld remote device with a simple quasi-intuitive operating structure. With this handset, elderly people will be able to control various household devices in their living environment. In order to develop an age-appropriate SP system, the aim of the study was to determine the requirements of elderly people and people with tremor. For this purpose, a mixed-method design, involving several assessments, a guideline-based interview, a task-based investigation and a questionnaire using a gesture catalog, was applied. The whole sample included 20 seniors being 60 years and older. In the process, qualitative requirements were collected on the topics of device use, operating problems, desired devices for gesture control, receiver unit, gestures, feedback and safety. The interview results emphasized the elderly participants’ needs to an easy and intuitive system use. Furthermore, concerns should be prioritized in order to the development of the system. In the quantitative evaluation, the use of various technical devices was analyzed and the frequency of used gestures was determined based the gesture catalog and the task-based investigation. The most frequently used gestures were horizontal, vertical, circular and targeting gestures. In summary, the elderly people were very interested in, and open-minded towards, the SP-system. In a comparison between healthy persons and persons with tremor, the results demonstrated only minimal differences regarding the requirements. |
BibTeX:
@inproceedings{Vorwerg2019,
author = {Vorwerg, S. and Eicher, C. and Ruser, H. and Piela, F. and Obée, F. and Kaltenbach, A. and Mechold, L.},
title = {Requirements for Gesture-Controlled Remote Operation to Facilitate Human-Technology Interaction in the Living Environment of Elderly People},
booktitle = {Human Aspects of IT for the Aged Population. Design for the Elderly and Technology Acceptance. HCII 2019.},
journal = {Lecture Notes in Computer Science},
publisher = {Springer},
school = {Universität der Bundeswehr München, Fakultät für Luft- und Raumfahrttechnik, LRT 2 - Institut für Angewandte Physik und Messtechnik, Professur: Dollinger, Günther},
year = {2019},
volume = {11592},
pages = {551--569},
editor = {Zhou J., Salvendy G.},
note = {including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics},
url = {https://link.springer.com/chapter/10.1007%2F978-3-030-22012-9_39},
doi = {https://doi.org/10.1007/978-3-030-22012-9_39}
}
|