2022
- Y. Chen, D. Harbecke, and L. Hennig, „Multilingual Relation Classification via Efficient and Effective Prompting,“ in Proceedings of EMNLP 2022, 2022.
[BibTeX] [Abstract]
Prompting pre-trained language models has achieved impressive performance on various NLP tasks, especially in low data regimes. Despite the success of prompting in monolingual settings, applying prompt-based methods in multilingual scenarios has been limited to a narrow set of tasks, due to the high cost of handcrafting multilingual prompts. In this paper, we present the first work on prompt-based multilingual relation classification (RC), by introducing an efficient and effective method that constructs prompts from relation triples and involves only minimal translation for the class labels. We evaluate its performance in fully supervised, few-shot and zero-shot scenarios, and analyze its effectiveness across 14 languages, prompt variants, and English-task training in cross-lingual settings. We find that in both fully supervised and few-shot scenarios, our prompt method beats competitive baselines: fine-tuning XLM-R EM, and null prompts. It also outperforms the random baseline by a large margin in zero-shot experiments. Our method requires little in-language knowledge and can be used as a strong baseline for similar multilingual classification tasks.
@inproceedings{chen-etal-2022-multilingual, title = "Multilingual Relation Classification via Efficient and Effective Prompting", author = "Chen, Yuxuan and Harbecke, David and Hennig, Leonhard", booktitle = "Proceedings of EMNLP 2022", year = "2022", abstract = "Prompting pre-trained language models has achieved impressive performance on various NLP tasks, especially in low data regimes. Despite the success of prompting in monolingual settings, applying prompt-based methods in multilingual scenarios has been limited to a narrow set of tasks, due to the high cost of handcrafting multilingual prompts. In this paper, we present the first work on prompt-based multilingual relation classification (RC), by introducing an efficient and effective method that constructs prompts from relation triples and involves only minimal translation for the class labels. We evaluate its performance in fully supervised, few-shot and zero-shot scenarios, and analyze its effectiveness across 14 languages, prompt variants, and English-task training in cross-lingual settings. We find that in both fully supervised and few-shot scenarios, our prompt method beats competitive baselines: fine-tuning XLM-R EM, and null prompts. It also outperforms the random baseline by a large margin in zero-shot experiments. Our method requires little in-language knowledge and can be used as a strong baseline for similar multilingual classification tasks.", } - Y. Chen, J. Mikkelsen, A. Binder, C. Alt, and L. Hennig, „A Comparative Study of Pre-trained Encoders for Low-Resource Named Entity Recognition,“ in Proceedings of the 7th Workshop on Representation Learning for NLP, Dublin, Ireland, 2022, p. 46–59. doi:10.18653/v1/2022.repl4nlp-1.6
[BibTeX] [Abstract] [Download PDF]
Pre-trained language models (PLM) are effective components of few-shot named entity recognition (NER) approaches when augmented with continued pre-training on task-specific out-of-domain data or fine-tuning on in-domain data. However, their performance in low-resource scenarios, where such data is not available, remains an open question. We introduce an encoder evaluation framework, and use it to systematically compare the performance of state-of-the-art pre-trained representations on the task of low-resource NER. We analyze a wide range of encoders pre-trained with different strategies, model architectures, intermediate-task fine-tuning, and contrastive learning. Our experimental results across ten benchmark NER datasets in English and German show that encoder performance varies significantly, suggesting that the choice of encoder for a specific low-resource scenario needs to be carefully evaluated.
@inproceedings{chen-etal-2022-comparative, title = "A Comparative Study of Pre-trained Encoders for Low-Resource Named Entity Recognition", author = "Chen, Yuxuan and Mikkelsen, Jonas and Binder, Arne and Alt, Christoph and Hennig, Leonhard", booktitle = "Proceedings of the 7th Workshop on Representation Learning for NLP", month = may, year = "2022", address = "Dublin, Ireland", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2022.repl4nlp-1.6", doi = "10.18653/v1/2022.repl4nlp-1.6", pages = "46--59", abstract = "Pre-trained language models (PLM) are effective components of few-shot named entity recognition (NER) approaches when augmented with continued pre-training on task-specific out-of-domain data or fine-tuning on in-domain data. However, their performance in low-resource scenarios, where such data is not available, remains an open question. We introduce an encoder evaluation framework, and use it to systematically compare the performance of state-of-the-art pre-trained representations on the task of low-resource NER. We analyze a wide range of encoders pre-trained with different strategies, model architectures, intermediate-task fine-tuning, and contrastive learning. Our experimental results across ten benchmark NER datasets in English and German show that encoder performance varies significantly, suggesting that the choice of encoder for a specific low-resource scenario needs to be carefully evaluated.", } - Y. Chen, „MEffi-Prompt: Multilingual Relation Classification via Efficient and Effective Prompting,“ FU Berlin, Master Thesis , 2022.
[BibTeX]@techreport{chen_2022_meffi, type = "Master Thesis", title = "MEffi-Prompt: Multilingual Relation Classification via Efficient and Effective Prompting", institution = "FU Berlin", author = "Chen, Yuxuan", year = "2022", } - D. Harbecke, Y. Chen, L. Hennig, and C. Alt, „Why only Micro-F1? Class Weighting of Measures for Relation Classification,“ in Proceedings of NLP Power! The First Workshop on Efficient Benchmarking in NLP, Dublin, Ireland, 2022, p. 32–41. doi:10.18653/v1/2022.nlppower-1.4
[BibTeX] [Abstract] [Download PDF]
Relation classification models are conventionally evaluated using only a single measure, e.g., micro-F1, macro-F1 or AUC. In this work, we analyze weighting schemes, such as micro and macro, for imbalanced datasets. We introduce a framework for weighting schemes, where existing schemes are extremes, and two new intermediate schemes. We show that reporting results of different weighting schemes better highlights strengths and weaknesses of a model.
@inproceedings{harbecke-etal-2022-micro, title = "Why only Micro-F1? Class Weighting of Measures for Relation Classification", author = "Harbecke, David and Chen, Yuxuan and Hennig, Leonhard and Alt, Christoph", booktitle = "Proceedings of NLP Power! The First Workshop on Efficient Benchmarking in NLP", month = may, year = "2022", address = "Dublin, Ireland", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2022.nlppower-1.4", doi = "10.18653/v1/2022.nlppower-1.4", pages = "32--41", abstract = "Relation classification models are conventionally evaluated using only a single measure, e.g., micro-F1, macro-F1 or AUC. In this work, we analyze weighting schemes, such as micro and macro, for imbalanced datasets. We introduce a framework for weighting schemes, where existing schemes are extremes, and two new intermediate schemes. We show that reporting results of different weighting schemes better highlights strengths and weaknesses of a model.", } - E. Mohammedi, „Neural information extraction in the renewable energy domain,“ TU Berlin, Master Thesis , 2022.
[BibTeX]@techreport{mohammedi_2022_renewable, type = "Master Thesis", title = "Neural information extraction in the renewable energy domain", institution = "TU Berlin", author = "Mohammedi, Elias", year = "2022", } - A. Taha, L. Hennig, and P. Knoth, „SelectiveMax: Enhancing NN model accuracy and accuracy balance using distribution-based activation for the logit layer,“ in Under review, 2022.
[BibTeX] [Abstract]
In classification using neural network, the output layer of the network (logit layer), contains values representing the likelihoods that an object belong to the corresponding class. The common way to convert this layer to a multi-class classification is to apply the softmax function and select the class with the maximum softmax value. In this paper, we propose a novel method for selecting a class based on the logits, which we call \textit{SelectiveMax}. The novel method significantly outperforms softmax both in precision and recall as well as class accuracy balance. In this method, before the logits are interpreted, they are weighted, based on a statistical model learned from the training set. As a result, the selected class is not necessarily the one with the maximum logit. We first show that the distributions of the logits corresponding to true classes as well as for false classes depend only on the model and not on the data. Based on this fact, we derive a logit weighting function that we call \textit{SelectiveMax}, which replaces the Softmax in building multi-class classification. SelectiveMax has been tested on 18 tasks in different domains and found to outperform the Softmax function in both accuracy and class accuracy balance.
@inproceedings{taha-etal-2022-selective, title = "SelectiveMax: Enhancing NN model accuracy and accuracy balance using distribution-based activation for the logit layer", author = "Taha, Aziz and Hennig, Leonhard and Knoth, Petr", booktitle = "Under review", year = "2022", abstract = "In classification using neural network, the output layer of the network (logit layer), contains values representing the likelihoods that an object belong to the corresponding class. The common way to convert this layer to a multi-class classification is to apply the softmax function and select the class with the maximum softmax value. In this paper, we propose a novel method for selecting a class based on the logits, which we call \textit{SelectiveMax}. The novel method significantly outperforms softmax both in precision and recall as well as class accuracy balance. In this method, before the logits are interpreted, they are weighted, based on a statistical model learned from the training set. As a result, the selected class is not necessarily the one with the maximum logit. We first show that the distributions of the logits corresponding to true classes as well as for false classes depend only on the model and not on the data. Based on this fact, we derive a logit weighting function that we call \textit{SelectiveMax}, which replaces the Softmax in building multi-class classification. SelectiveMax has been tested on 18 tasks in different domains and found to outperform the Softmax function in both accuracy and class accuracy balance. ", }
2021
- S. Akarshe, „Machine Learning Entities for Food Image Detection and quantifying its Nutritional Components,“ SRH University of Applied Sciences Berlin, Master Thesis , 2021.
[BibTeX]@techreport{akarshe_2021_food, type = "Master Thesis", title = "Machine Learning Entities for Food Image Detection and quantifying its Nutritional Components", institution = "SRH University of Applied Sciences Berlin", author = "Akarshe, Sayli", year = "2021", } - B. van Aken, J. Papaioannou, M. Mayrdorfer, K. Budde, F. Gers, and A. Loeser, „Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration,“ in Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, Online, 2021, p. 881–893.
[BibTeX] [Abstract] [Download PDF]
Outcome prediction from clinical text can prevent doctors from overlooking possible risks and help hospitals to plan capacities. We simulate patients at admission time, when decision support can be especially valuable, and contribute a novel *admission to discharge* task with four common outcome prediction targets: Diagnoses at discharge, procedures performed, in-hospital mortality and length-of-stay prediction. The ideal system should infer outcomes based on symptoms, pre-conditions and risk factors of a patient. We evaluate the effectiveness of language models to handle this scenario and propose *clinical outcome pre-training* to integrate knowledge about patient outcomes from multiple public sources. We further present a simple method to incorporate ICD code hierarchy into the models. We show that our approach improves performance on the outcome tasks against several baselines. A detailed analysis reveals further strengths of the model, including transferability, but also weaknesses such as handling of vital values and inconsistencies in the underlying data.
@inproceedings{van-aken-etal-2021-clinical, title = "Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration", author = "van Aken, Betty and Papaioannou, Jens-Michalis and Mayrdorfer, Manuel and Budde, Klemens and Gers, Felix and Loeser, Alexander", booktitle = "Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume", month = apr, year = "2021", address = "Online", publisher = "Association for Computational Linguistics", url = "https://www.aclweb.org/anthology/2021.eacl-main.75", pages = "881--893", abstract = "Outcome prediction from clinical text can prevent doctors from overlooking possible risks and help hospitals to plan capacities. We simulate patients at admission time, when decision support can be especially valuable, and contribute a novel *admission to discharge* task with four common outcome prediction targets: Diagnoses at discharge, procedures performed, in-hospital mortality and length-of-stay prediction. The ideal system should infer outcomes based on symptoms, pre-conditions and risk factors of a patient. We evaluate the effectiveness of language models to handle this scenario and propose *clinical outcome pre-training* to integrate knowledge about patient outcomes from multiple public sources. We further present a simple method to incorporate ICD code hierarchy into the models. We show that our approach improves performance on the outcome tasks against several baselines. A detailed analysis reveals further strengths of the model, including transferability, but also weaknesses such as handling of vital values and inconsistencies in the underlying data.", } - S. Castle, R. Schwarzenberg, and M. Pourvali, „Detecting Covariate Drift with Explanations,“ in Natural Language Processing and Chinese Computing, Cham, 2021, p. 317–322.
[BibTeX] [Abstract]
Detecting when there is a domain drift between training and inference data is important for any model evaluated on data collected in real time. Many current data drift detection methods only utilize input features to detect domain drift. While effective, these methods disregard the model’s evaluation of the data, which may be a significant source of information about the data domain. We propose to use information from the model in the form of explanations, specifically gradient times input, in order to utilize this information. Following the framework of Rabanser et al. [11], we combine these explanations with two-sample tests in order to detect a shift in distribution between training and evaluation data. Promising initial experiments show that explanations provide useful information for detecting shift, which potentially improves upon the current state-of-the-art.
@InProceedings{10.1007/978-3-030-88483-3_24, author="Castle, Steffen and Schwarzenberg, Robert and Pourvali, Mohsen", editor="Wang, Lu and Feng, Yansong and Hong, Yu and He, Ruifang", title="Detecting Covariate Drift with Explanations", booktitle="Natural Language Processing and Chinese Computing", year="2021", publisher="Springer International Publishing", address="Cham", pages="317--322", abstract="Detecting when there is a domain drift between training and inference data is important for any model evaluated on data collected in real time. Many current data drift detection methods only utilize input features to detect domain drift. While effective, these methods disregard the model's evaluation of the data, which may be a significant source of information about the data domain. We propose to use information from the model in the form of explanations, specifically gradient times input, in order to utilize this information. Following the framework of Rabanser et al. [11], we combine these explanations with two-sample tests in order to detect a shift in distribution between training and evaluation data. Promising initial experiments show that explanations provide useful information for detecting shift, which potentially improves upon the current state-of-the-art.", isbn="978-3-030-88483-3" } - J. Frey and S. Hellmann, „FAIR Linked Data – Towards a Linked Data Backbone for Users and Machines,“ in Companion of The Web Conference 2021, Virtual Event / Ljubljana, Slovenia, April 19-23, 2021, 2021, p. 431–435. doi:10.1145/3442442.3451364
[BibTeX] [Download PDF]@inproceedings{DBLP:conf/www/FreyH21, author = {Johannes Frey and Sebastian Hellmann}, editor = {Jure Leskovec and Marko Grobelnik and Marc Najork and Jie Tang and Leila Zia}, title = {{FAIR} Linked Data - Towards a Linked Data Backbone for Users and Machines}, booktitle = {Companion of The Web Conference 2021, Virtual Event / Ljubljana, Slovenia, April 19-23, 2021}, pages = {431--435}, publisher = {{ACM} / {IW3C2}}, year = {2021}, url = {https://doi.org/10.1145/3442442.3451364}, doi = {10.1145/3442442.3451364}, timestamp = {Mon, 07 Jun 2021 14:34:36 +0200}, biburl = {https://dblp.org/rec/conf/www/FreyH21.bib}, bibsource = {dblp computer science bibliography, https://dblp.org} } - S. Hellmann, J. Frey, M. Hofer, M. Dojchinovski, K. Wecel, and W. Lewoniewski, „Towards a Systematic Approach to Sync Factual Data across Wikipedia, Wikidata and External Data Sources,“ in Proceedings of the Conference on Digital Curation Technologies (Qurator 2021), Berlin, Germany, February 8th – to – 12th, 2021, 2021.
[BibTeX] [Download PDF]@inproceedings{DBLP:conf/qurator/HellmannFHDWL21, author = {Sebastian Hellmann and Johannes Frey and Marvin Hofer and Milan Dojchinovski and Krzysztof Wecel and Wlodzimierz Lewoniewski}, editor = {Adrian Paschke and Georg Rehm and Jamal Al Qundus and Clemens Neudecker and Lydia Pintscher}, title = {Towards a Systematic Approach to Sync Factual Data across Wikipedia, Wikidata and External Data Sources}, booktitle = {Proceedings of the Conference on Digital Curation Technologies (Qurator 2021), Berlin, Germany, February 8th - to - 12th, 2021}, series = {{CEUR} Workshop Proceedings}, volume = {2836}, publisher = {CEUR-WS.org}, year = {2021}, url = {http://ceur-ws.org/Vol-2836/qurator2021\_paper\_18.pdf}, timestamp = {Wed, 28 Apr 2021 17:11:52 +0200}, biburl = {https://dblp.org/rec/conf/qurator/HellmannFHDWL21.bib}, bibsource = {dblp computer science bibliography, https://dblp.org} } - L. Hennig, P. T. Truong, and A. Gabryszak, „MobIE: A German Dataset for Named Entity Recognition, Entity Linking and Relation Extraction in the Mobility Domain,“ in Proceedings of the 17th Conference on Natural Language Processing (KONVENS 2021), Düsseldorf, Germany, 2021, p. 223–227.
[BibTeX] [Download PDF]@inproceedings{hennig-etal-2021-mobie, title = "{M}ob{IE}: A {G}erman Dataset for Named Entity Recognition, Entity Linking and Relation Extraction in the Mobility Domain", author = "Hennig, Leonhard and Truong, Phuc Tran and Gabryszak, Aleksandra", booktitle = "Proceedings of the 17th Conference on Natural Language Processing (KONVENS 2021)", month = "6--9 " # sep, year = "2021", address = {D{\"u}sseldorf, Germany}, publisher = "KONVENS 2021 Organizers", url = "https://aclanthology.org/2021.konvens-1.22", pages = "223--227", } - L. Hennig and H. Uszkoreit, „Semantische Technologien für Enterprise Intelligence am Beispiel von Lieferkettenbeobachtung,“ in Semantische Datenintelligenz im Einsatz, B. Ege and A. Paschke, Eds., Springer Vieweg, 2021, p. 87–126.
[BibTeX]@incollection{hennig_2021_semantische_technologien, author = {Hennig, Leonhard and Uszkoreit, Hans}, editor = {Ege, B\"{o}rte\c{c}in and Paschke, Adrian}, title = {Semantische Technologien für Enterprise Intelligence am Beispiel von Lieferkettenbeobachtung}, booktitle = {Semantische Datenintelligenz im Einsatz}, year = {2021}, pages = {87--126}, publisher = {Springer Vieweg}, isbn = {978-3-658-31938-0} } - J. Papaioannou, M. Mayrdorfer, S. Arnold, F. A. Gers, K. Budde, and A. Löser, „Aspect-Based Passage Retrieval with Contextualized Discourse Vectors,“ in Advances in Information Retrieval, Cham, 2021, p. 537–542.
[BibTeX] [Abstract]
Passage retrieval is the task of retrieving only the portions of a document that are relevant to a particular information need. One application medical doctors and researchers face is the challenge of reading a large amount of novel literature. For example, since the outbreak of Coronavirus disease 2019 (COVID-19), tens of thousands of papers have been published each month about the disease. We demonstrate how we can support healthcare professionals in this exploratory research task with our neural passage retrieval system based on Contextualized Discourse Vectors (CDV). CDV captures the discourse of long documents on sentence level and allows to query a large corpus with medical entities and aspects. Our demonstration covers over 27,000 diseases and 14,000 clinical aspects including symptoms, diagnostics, treatments and medications. It returns passages and highlights sentences to effectively answer clinical queries with up to 65{\%} Recall@1. We showcase our system on the COVID-19 Open Research Dataset (CORD-19), Orphanet and Wikipedia diseases corpora.
@InProceedings{10.1007/978-3-030-72240-1_61, author="Papaioannou, Jens-Michalis and Mayrdorfer, Manuel and Arnold, Sebastian and Gers, Felix A. and Budde, Klemens and L{\"o}ser, Alexander", editor="Hiemstra, Djoerd and Moens, Marie-Francine and Mothe, Josiane and Perego, Raffaele and Potthast, Martin and Sebastiani, Fabrizio", title="Aspect-Based Passage Retrieval with Contextualized Discourse Vectors", booktitle="Advances in Information Retrieval", year="2021", publisher="Springer International Publishing", address="Cham", pages="537--542", abstract="Passage retrieval is the task of retrieving only the portions of a document that are relevant to a particular information need. One application medical doctors and researchers face is the challenge of reading a large amount of novel literature. For example, since the outbreak of Coronavirus disease 2019 (COVID-19), tens of thousands of papers have been published each month about the disease. We demonstrate how we can support healthcare professionals in this exploratory research task with our neural passage retrieval system based on Contextualized Discourse Vectors (CDV). CDV captures the discourse of long documents on sentence level and allows to query a large corpus with medical entities and aspects. Our demonstration covers over 27,000 diseases and 14,000 clinical aspects including symptoms, diagnostics, treatments and medications. It returns passages and highlights sentences to effectively answer clinical queries with up to 65{\%} Recall@1. We showcase our system on the COVID-19 Open Research Dataset (CORD-19), Orphanet and Wikipedia diseases corpora.", isbn="978-3-030-72240-1" } - A. Taha, L. Hennig, and P. Knoth, „Confidence estimation of classification based on the distribution of the neural network output layer,“ in Under review, 2021.
[BibTeX] [Abstract]
One of the most common problems preventing the application of prediction models in the real world is lack of generalization: The accuracy of models, measured in the benchmark does repeat itself on future data, e.g. in the settings of real business. There is relatively little methods exist that estimate the confidence of prediction models. In this paper, we propose novel methods that, given a neural network classification model, estimate uncertainty of particular predictions generated by this model. Furthermore, we propose a method that, given a model and a confidence level, calculates a threshold that separates prediction generated by this model into two subsets, one of them meets the given confidence level. In contrast to other methods, the proposed methods do not require any changes on existing neural networks, because they simply build on the output logit layer of a common neural network. In particular, the methods infer the confidence of a particular prediction based on the distribution of the logit values corresponding to this prediction. The proposed methods constitute a tool that is recommended for filtering predictions in the process of knowledge extraction, e.g. based on web scrapping, where predictions subsets are identified that maximize the precision on cost of the recall, which is less important due to the availability of data. The method has been tested on different tasks including relation extraction, named entity recognition and image classification to show the significant increase of accuracy achieved.
@inproceedings{taha-etal-2021-confidence, title = "Confidence estimation of classification based on the distribution of the neural network output layer", author = "Taha, Aziz and Hennig, Leonhard and Knoth, Petr", booktitle = "Under review", year = "2021", abstract = "One of the most common problems preventing the application of prediction models in the real world is lack of generalization: The accuracy of models, measured in the benchmark does repeat itself on future data, e.g. in the settings of real business. There is relatively little methods exist that estimate the confidence of prediction models. In this paper, we propose novel methods that, given a neural network classification model, estimate uncertainty of particular predictions generated by this model. Furthermore, we propose a method that, given a model and a confidence level, calculates a threshold that separates prediction generated by this model into two subsets, one of them meets the given confidence level. In contrast to other methods, the proposed methods do not require any changes on existing neural networks, because they simply build on the output logit layer of a common neural network. In particular, the methods infer the confidence of a particular prediction based on the distribution of the logit values corresponding to this prediction. The proposed methods constitute a tool that is recommended for filtering predictions in the process of knowledge extraction, e.g. based on web scrapping, where predictions subsets are identified that maximize the precision on cost of the recall, which is less important due to the availability of data. The method has been tested on different tasks including relation extraction, named entity recognition and image classification to show the significant increase of accuracy achieved. ", } - U. Yaseen and S. Langer, „Neural Text Classification and Stacked Heterogeneous Embeddings for Named Entity Recognition in SMM4H 2021,“ in Proceedings of the Sixth Social Media Mining for Health (\#SMM4H) Workshop and Shared Task, Mexico City, Mexico, 2021, p. 83–87. doi:10.18653/v1/2021.smm4h-1.14
[BibTeX] [Abstract]
This paper presents our findings from participating in the SMM4H Shared Task 2021. We addressed Named Entity Recognition (NER) and Text Classification. To address NER we explored BiLSTM-CRF with Stacked Heterogeneous embeddings and linguistic features. We investigated various machine learning algorithms (logistic regression, SVM and Neural Networks) to address text classification. Our proposed approaches can be generalized to different languages and we have shown its effectiveness for English and Spanish. Our text classification submissions have achieved competitive performance with F1-score of 0.46 and 0.90 on ADE Classification (Task 1a) and Profession Classification (Task 7a) respectively. In the case of NER, our submissions scored F1-score of 0.50 and 0.82 on ADE Span Detection (Task 1b) and Profession span detection (Task 7b) respectively.
@inproceedings{yaseen-langer-2021-neural, title = "Neural Text Classification and Stacked Heterogeneous Embeddings for Named Entity Recognition in {SMM}4{H} 2021", author = "Yaseen, Usama and Langer, Stefan", booktitle = "Proceedings of the Sixth Social Media Mining for Health ({\#}SMM4H) Workshop and Shared Task", month = jun, year = "2021", address = "Mexico City, Mexico", publisher = "Association for Computational Linguistics", url = https://www.aclweb.org/anthology/2021.smm4h-1.14, doi = "10.18653/v1/2021.smm4h-1.14", pages = "83--87", abstract = "This paper presents our findings from participating in the SMM4H Shared Task 2021. We addressed Named Entity Recognition (NER) and Text Classification. To address NER we explored BiLSTM-CRF with Stacked Heterogeneous embeddings and linguistic features. We investigated various machine learning algorithms (logistic regression, SVM and Neural Networks) to address text classification. Our proposed approaches can be generalized to different languages and we have shown its effectiveness for English and Spanish. Our text classification submissions have achieved competitive performance with F1-score of 0.46 and 0.90 on ADE Classification (Task 1a) and Profession Classification (Task 7a) respectively. In the case of NER, our submissions scored F1-score of 0.50 and 0.82 on ADE Span Detection (Task 1b) and Profession span detection (Task 7b) respectively.", }
2020
- C. Alt, A. Gabryszak, and L. Hennig, „Probing Linguistic Features of Sentence-Level Representations in Neural Relation Extraction,“ in Proceedings of ACL 2020, 2020.
[BibTeX] [Abstract] [Download PDF]
Despite the recent progress, little is known about the features captured by state-of-the-art neural relation extraction (RE) models. Common methods encode the source sentence, conditioned on the entity mentions, before classifying the relation. However, the complexity of the task makes it difficult to understand how encoder architecture and supporting linguistic knowledge affect the features learned by the encoder. We introduce 14 probing tasks targeting linguistic properties relevant to RE, and we use them to study representations learned by more than 40 different encoder architecture and linguistic feature combinations trained on two datasets, TACRED and SemEval 2010 Task 8. We find that the bias induced by the architecture and the inclusion of linguistic features are clearly expressed in the probing task performance. For example, adding contextualized word representations greatly increases performance on probing tasks with a focus on named entity and part-of-speech information, and yields better results in RE. In contrast, entity masking improves RE, but considerably lowers performance on entity type related probing tasks.
@inproceedings{Alt2020ProbingLF, title={Probing Linguistic Features of Sentence-Level Representations in Neural Relation Extraction}, author={Christoph Alt and Aleksandra Gabryszak and Leonhard Hennig}, abstract={Despite the recent progress, little is known about the features captured by state-of-the-art neural relation extraction (RE) models. Common methods encode the source sentence, conditioned on the entity mentions, before classifying the relation. However, the complexity of the task makes it difficult to understand how encoder architecture and supporting linguistic knowledge affect the features learned by the encoder. We introduce 14 probing tasks targeting linguistic properties relevant to RE, and we use them to study representations learned by more than 40 different encoder architecture and linguistic feature combinations trained on two datasets, TACRED and SemEval 2010 Task 8. We find that the bias induced by the architecture and the inclusion of linguistic features are clearly expressed in the probing task performance. For example, adding contextualized word representations greatly increases performance on probing tasks with a focus on named entity and part-of-speech information, and yields better results in RE. In contrast, entity masking improves RE, but considerably lowers performance on entity type related probing tasks.}, booktitle={Proceedings of ACL 2020}, year={2020}, url={https://arxiv.org/abs/2004.08134}, publisher = {Association for Computational Linguistics}, } - C. Alt, A. Gabryszak, and L. Hennig, „TACRED Revisited: A Thorough Evaluation of the TACRED Relation Extraction Task,“ in Proceedings of ACL 2020, 2020.
[BibTeX] [Abstract] [Download PDF]
TACRED (Zhang et al., 2017) is one of the largest, most widely used crowdsourced datasets in Relation Extraction (RE). But, even with recent advances in unsupervised pre-training and knowledge enhanced neural RE, models still show a high error rate. In this paper, we investigate the questions: Have we reached a performance ceiling or is there still room for improvement? And how do crowd annotations, dataset, and models contribute to this error rate? To answer these questions, we first validate the most challenging 5K examples in the development and test sets using trained annotators. We find that label errors account for 8% absolute F1 test error, and that more than 50% of the examples need to be relabeled. On the relabeled test set the average F1 score of a large baseline model set improves from 62.1 to 70.1. After validation, we analyze misclassifications on the challenging instances, categorize them into linguistically motivated error groups, and verify the resulting error hypotheses on three state-of-the-art RE models. We show that two groups of ambiguous relations are responsible for most of the remaining errors and that models may adopt shallow heuristics on the dataset when entities are not masked.
@inproceedings{Alt2020TACREDRA, title={TACRED Revisited: A Thorough Evaluation of the TACRED Relation Extraction Task}, author={Christoph Alt and Aleksandra Gabryszak and Leonhard Hennig}, booktitle={Proceedings of ACL 2020}, abstract={TACRED (Zhang et al., 2017) is one of the largest, most widely used crowdsourced datasets in Relation Extraction (RE). But, even with recent advances in unsupervised pre-training and knowledge enhanced neural RE, models still show a high error rate. In this paper, we investigate the questions: Have we reached a performance ceiling or is there still room for improvement? And how do crowd annotations, dataset, and models contribute to this error rate? To answer these questions, we first validate the most challenging 5K examples in the development and test sets using trained annotators. We find that label errors account for 8% absolute F1 test error, and that more than 50% of the examples need to be relabeled. On the relabeled test set the average F1 score of a large baseline model set improves from 62.1 to 70.1. After validation, we analyze misclassifications on the challenging instances, categorize them into linguistically motivated error groups, and verify the resulting error hypotheses on three state-of-the-art RE models. We show that two groups of ambiguous relations are responsible for most of the remaining errors and that models may adopt shallow heuristics on the dataset when entities are not masked. }, year={2020}, url={https://arxiv.org/abs/2004.14855}, publisher = {Association for Computational Linguistics}, } - S. Arnold, B. van Aken, P. Grundmann, F. A. Gers, and A. Löser, „Learning Contextualized Document Representations for Healthcare Answer Retrieval,“ in WWW ’20: The Web Conference 2020, Taipei, Taiwan, April 20-24, 2020, 2020, p. 1332–1343. doi:10.1145/3366423.3380208
[BibTeX] [Download PDF]@inproceedings{DBLP:conf/www/0001AGGL20, author = {Sebastian Arnold and Betty van Aken and Paul Grundmann and Felix A. Gers and Alexander L{\"{o}}ser}, editor = {Yennun Huang and Irwin King and Tie{-}Yan Liu and Maarten van Steen}, title = {Learning Contextualized Document Representations for Healthcare Answer Retrieval}, booktitle = {{WWW} '20: The Web Conference 2020, Taipei, Taiwan, April 20-24, 2020}, pages = {1332--1343}, publisher = {{ACM} / {IW3C2}}, year = {2020}, url = {https://doi.org/10.1145/3366423.3380208}, doi = {10.1145/3366423.3380208}, timestamp = {Wed, 06 May 2020 12:56:16 +0200}, biburl = {https://dblp.org/rec/conf/www/0001AGGL20.bib}, bibsource = {dblp computer science bibliography, https://dblp.org} } - Y. Chaudhary, H. Schütze, and P. Gupta, „Explainable and Discourse Topic-aware Neural Language Understanding,“ in Proceedings of the 37th International Conference on Machine Learning, ICML, virtual, 2020.
[BibTeX]@inproceedings{chaudhary-etal-2020-nclm-icml, title = "Explainable and Discourse Topic-aware Neural Language Understanding", author = {Chaudhary, Yatin and Sch{\"u}tze, Hinrich and Gupta, Pankaj}, booktitle = "Proceedings of the 37th International Conference on Machine Learning, ICML", month = jul, year = "2020", address = "virtual" } - J. Frey, D. Streitmatter, F. Götz, and S. Hellmann, „DBpedia Archivo – A Web-Scale Interface for Ontology Archiving under Consumer-oriented Aspects,“ submitted to Semantics 2020, 2020.
[BibTeX] [Download PDF]@article{freydbpedia, title = {DBpedia Archivo - A Web-Scale Interface for Ontology Archiving under Consumer-oriented Aspects}, added-at = {2020-06-29T15:55:45.000+0200}, author = {Frey, Johannes and Streitmatter, Denis and G{\"o}tz, Fabian and Hellmann, Sebastian}, biburl = {https://www.bibsonomy.org/bibtex/25f8565541c6412590594b898a632fc0a/aksw}, interhash = {2836caa18d470561ec3169b3f59aaf21}, intrahash = {5f8565541c6412590594b898a632fc0a}, journal = {submitted to Semantics 2020}, keywords = {arndt frey group_aksw hellmann kilt streitmatter}, timestamp = {2020-06-29T15:55:45.000+0200}, url = {https://svn.aksw.org/papers/2020/semantics_archivo/public.pdf}, year = 2020 } - P. Gupta, Y. Chaudhary, T. Runkler, and H. Schütze, „Neural Topic Modeling with Continual Lifelong Learning,“ in Proceedings of the 37th International Conference on Machine Learning, ICML, virtual, 2020.
[BibTeX]@inproceedings{gupta-etal-2020-lntm-icml, title = "Neural Topic Modeling with Continual Lifelong Learning", author = {Gupta, Pankaj and Chaudhary, Yatin and Runkler, Thomas and Sch{\"u}tze, Hinrich}, booktitle = "Proceedings of the 37th International Conference on Machine Learning, ICML", month = jul, year = "2020", address = "virtual" } - M. Hofer, S. Hellmann, M. Dojchinovski, and J. Frey, „The New DBpedia Release Cycle: Increasing Agility and Efficiency in Knowledge Extraction Workflows,“ submitted to Semantics 2020, vol. 16, 2020.
[BibTeX] [Download PDF]@article{hoferdbpedai, title = {The New DBpedia Release Cycle: Increasing Agility and Efficiency in Knowledge Extraction Workflows}, added-at = {2020-06-29T15:55:45.000+0200}, author = {Hofer, Marvin and Hellmann, Sebastian and Dojchinovski, Milan and Frey, Johannes}, biburl = {https://www.bibsonomy.org/bibtex/2bbb6a227c9fdcd96b332132877226b6f/aksw}, booktitle = {16th International Conference on Semantic Systems}, interhash = {592b96964e8c91d044c175c1708d2d34}, intrahash = {bbb6a227c9fdcd96b332132877226b6f}, journal = {submitted to Semantics 2020}, keywords = {frey group_aksw hellmann hofer kilt}, publisher = {Springer}, series = {LNCS}, timestamp = {2020-06-29T15:55:45.000+0200}, url = {https://svn.aksw.org/papers/2020/semantics_marvin/public.pdf}, volume = 16, year = 2020 } - M. Hübner, C. Alt, R. Schwarzenberg, and L. Hennig, „Defx at SemEval-2020 Task 6: Joint Extraction of Concepts and Relations for Definition Extraction,“ in Proceedings of the Fourteenth Workshop on Semantic Evaluation, Barcelona (online), 2020, p. 704–709.
[BibTeX] [Download PDF]@inproceedings{defx_at_semeval, title = "Defx at {S}em{E}val-2020 Task 6: Joint Extraction of Concepts and Relations for Definition Extraction", author = {H{\"u}bner, Marc and Alt, Christoph and Schwarzenberg, Robert and Hennig, Leonhard}, booktitle = "Proceedings of the Fourteenth Workshop on Semantic Evaluation", month = dec, year = "2020", address = "Barcelona (online)", publisher = "International Committee for Computational Linguistics", url = "https://www.aclweb.org/anthology/2020.semeval-1.92", pages = "704--709" } - K. Lazaridou, A. Löser, M. Mestre, and F. Naumann, „Discovering Biased News Articles Leveraging Multiple Human Annotations,“ in Proceedings of the 12th Language Resources and Evaluation Conference, Marseille, France, 2020, p. 1268–1277.
[BibTeX] [Abstract] [Download PDF]
Unbiased and fair reporting is an integral part of ethical journalism. Yet, political propaganda and one-sided views can be found in the news and can cause distrust in media. Both accidental and deliberate political bias affect the readers and shape their views. We contribute to a trustworthy media ecosystem by automatically identifying politically biased news articles. We introduce novel corpora annotated by two communities, i.e., domain experts and crowd workers, and we also consider automatic article labels inferred by the newspapers{‚} ideologies. Our goal is to compare domain experts to crowd workers and also to prove that media bias can be detected automatically. We classify news articles with a neural network and we also improve our performance in a self-supervised manner.
@inproceedings{lazaridou-etal-2020-discovering, title = "Discovering Biased News Articles Leveraging Multiple Human Annotations", author = {Lazaridou, Konstantina and L{\"o}ser, Alexander and Mestre, Maria and Naumann, Felix}, booktitle = "Proceedings of the 12th Language Resources and Evaluation Conference", month = may, year = "2020", address = "Marseille, France", publisher = "European Language Resources Association", url = "https://www.aclweb.org/anthology/2020.lrec-1.159", pages = "1268--1277", abstract = "Unbiased and fair reporting is an integral part of ethical journalism. Yet, political propaganda and one-sided views can be found in the news and can cause distrust in media. Both accidental and deliberate political bias affect the readers and shape their views. We contribute to a trustworthy media ecosystem by automatically identifying politically biased news articles. We introduce novel corpora annotated by two communities, i.e., domain experts and crowd workers, and we also consider automatic article labels inferred by the newspapers{'} ideologies. Our goal is to compare domain experts to crowd workers and also to prove that media bias can be detected automatically. We classify news articles with a neural network and we also improve our performance in a self-supervised manner.", language = "English", ISBN = "979-10-95546-34-4", } - T. Oberhauser, T. Bischoff, K. Brendel, M. Menke, T. Klatt, A. Siu, F. A. Gers, and A. Löser, „TrainX – Named Entity Linking with Active Sampling and Bi-Encoders,“ in Proceedings of the 28th International Conference on Computational Linguistics: System Demonstrations, Barcelona, Spain (Online), 2020, p. 64–69. doi:10.18653/v1/2020.coling-demos.12
[BibTeX] [Abstract] [Download PDF]
We demonstrate TrainX, a system for Named Entity Linking for medical experts. It combines state-of-the-art entity recognition and linking architectures, such as Flair and fine-tuned Bi-Encoders based on BERT, with an easy-to-use interface for healthcare professionals. We support medical experts in annotating training data by using active sampling strategies to forward informative samples to the annotator. We demonstrate that our model is capable of linking against large knowledge bases, such as UMLS (3.6 million entities), and supporting zero-shot cases, where the linker has never seen the entity before. Those zero-shot capabilities help to mitigate the problem of rare and expensive training data that is a common issue in the medical domain.
@inproceedings{oberhauser-etal-2020-trainx, title = "{T}rain{X} {--} Named Entity Linking with Active Sampling and Bi-Encoders", author = {Oberhauser, Tom and Bischoff, Tim and Brendel, Karl and Menke, Maluna and Klatt, Tobias and Siu, Amy and Gers, Felix Alexander and L{\"o}ser, Alexander}, booktitle = "Proceedings of the 28th International Conference on Computational Linguistics: System Demonstrations", month = dec, year = "2020", address = "Barcelona, Spain (Online)", publisher = "International Committee on Computational Linguistics (ICCL)", url = "https://www.aclweb.org/anthology/2020.coling-demos.12", doi = "10.18653/v1/2020.coling-demos.12", pages = "64--69", abstract = "We demonstrate TrainX, a system for Named Entity Linking for medical experts. It combines state-of-the-art entity recognition and linking architectures, such as Flair and fine-tuned Bi-Encoders based on BERT, with an easy-to-use interface for healthcare professionals. We support medical experts in annotating training data by using active sampling strategies to forward informative samples to the annotator. We demonstrate that our model is capable of linking against large knowledge bases, such as UMLS (3.6 million entities), and supporting zero-shot cases, where the linker has never seen the entity before. Those zero-shot capabilities help to mitigate the problem of rare and expensive training data that is a common issue in the medical domain.", } - R. Schneider, T. Oberhauser, P. Grundmann, F. A. Gers, A. Löser, and S. Staab, „Is Language Modeling Enough? Evaluating Effective Embedding Combinations,“ in Proceedings of the 12th Language Resources and Evaluation Conference., Marseille, France, 2020.
[BibTeX]@inproceedings{schneider2020language, title = {Is Language Modeling Enough? Evaluating Effective Embedding Combinations}, booktitle = {Proceedings of the 12th Language Resources and Evaluation Conference.}, author = {Schneider, Rudolf and Oberhauser, Tom and Grundmann, Paul and Gers, Felix Alexander and Löser, Alexander and Staab, Steffen}, year = {2020}, month = May, volume = {12}, publisher = {ELRA}, address = {Marseille, France} } - H. Zhang, L. Hennig, C. Alt, C. Hu, Y. Meng, and C. Wang, „Bootstrapping Named Entity Recognition in E-Commerce with Positive Unlabeled Learning,“ in Proceedings of the Third Workshop on e-Commerce and NLP, Seattle, Washington, 2020.
[BibTeX] [Abstract]
Named Entity Recognition (NER) in domains like e-commerce is an understudied problem due to the lack of annotated datasets. Recognizing novel entity types in this domain, such as products, components, and attributes, is challenging because of their linguistic complexity and the low coverage of existing knowledge resources. To address this problem, we present a bootstrapped positive-unlabeled learning algorithm that integrates domain-specific linguistic features to quickly and efficiently expand the seed dictionary. The model achieves an average F1 score of 72.02\% on a novel dataset of product descriptions, an improvement of 3.6{\textbackslash}\% over a baseline BiLSTM classifier, and in particular exhibits better recall (4.96\% on average)
@inproceedings{zhang_bootstrapping_2020, address = {Seattle, Washington}, title = {Bootstrapping {Named} {Entity} {Recognition} in {E}-{Commerce} with {Positive} {Unlabeled} {Learning}}, abstract = {Named Entity Recognition (NER) in domains like e-commerce is an understudied problem due to the lack of annotated datasets. Recognizing novel entity types in this domain, such as products, components, and attributes, is challenging because of their linguistic complexity and the low coverage of existing knowledge resources. To address this problem, we present a bootstrapped positive-unlabeled learning algorithm that integrates domain-specific linguistic features to quickly and efficiently expand the seed dictionary. The model achieves an average F1 score of 72.02\% on a novel dataset of product descriptions, an improvement of 3.6{\textbackslash}\% over a baseline BiLSTM classifier, and in particular exhibits better recall (4.96\% on average)}, booktitle = {Proceedings of the {Third} {Workshop} on e-{Commerce} and {NLP}}, author = {Zhang, Hanchu and Hennig, Leonhard and Alt, Christoph and Hu, Changjian and Meng, Yao and Wang, Chao}, year = {2020} }
2019
- B. van Aken, B. Winter, A. Löser, and F. A. Gers, „How Does BERT Answer Questions?,“ Proceedings of the 28th ACM International Conference on Information and Knowledge Management – CIKM ’19, 2019.
[BibTeX]@article{van_Aken_2019, title={How Does BERT Answer Questions?}, journal={Proceedings of the 28th ACM International Conference on Information and Knowledge Management - CIKM ’19}, publisher={ACM Press}, author={van Aken, Betty and Winter, Benjamin and Löser, Alexander and Gers, Felix A.}, year={2019} } - C. Alt, M. Hübner, and L. Hennig, „Fine-tuning Pre-Trained Transformer Language Models to Distantly Supervised Relation Extraction,“ in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, 2019, p. 1388–1398.
[BibTeX] [Abstract] [Download PDF]
Distantly supervised relation extraction is widely used to extract relational facts from text, but suffers from noisy labels. Current relation extraction methods try to alleviate the noise by multi-instance learning and by providing supporting linguistic and contextual information to more efficiently guide the relation classification. While achieving state-of-the-art results, we observed these models to be biased towards recognizing a limited set of relations with high precision, while ignoring those in the long tail. To address this gap, we utilize a pre-trained language model, the OpenAI Generative Pre-trained Transformer (GPT) (Radford et al., 2018). The GPT and similar models have been shown to capture semantic and syntactic features, and also a notable amount of “common-sense” knowledge, which we hypothesize are important features for recognizing a more diverse set of relations. By extending the GPT to the distantly supervised setting, and fine-tuning it on the NYT10 dataset, we show that it predicts a larger set of distinct relation types with high confidence. Manual and automated evaluation of our model shows that it achieves a state-of-the-art AUC score of 0.422 on the NYT10 dataset, and performs especially well at higher recall levels.
@inproceedings{alt_fine-tuning_2019, address = {Florence, Italy}, title = {Fine-tuning {Pre}-{Trained} {Transformer} {Language} {Models} to {Distantly} {Supervised} {Relation} {Extraction}}, url = {https://www.aclweb.org/anthology/P19-1134}, abstract = {Distantly supervised relation extraction is widely used to extract relational facts from text, but suffers from noisy labels. Current relation extraction methods try to alleviate the noise by multi-instance learning and by providing supporting linguistic and contextual information to more efficiently guide the relation classification. While achieving state-of-the-art results, we observed these models to be biased towards recognizing a limited set of relations with high precision, while ignoring those in the long tail. To address this gap, we utilize a pre-trained language model, the OpenAI Generative Pre-trained Transformer (GPT) (Radford et al., 2018). The GPT and similar models have been shown to capture semantic and syntactic features, and also a notable amount of “common-sense” knowledge, which we hypothesize are important features for recognizing a more diverse set of relations. By extending the GPT to the distantly supervised setting, and fine-tuning it on the NYT10 dataset, we show that it predicts a larger set of distinct relation types with high confidence. Manual and automated evaluation of our model shows that it achieves a state-of-the-art AUC score of 0.422 on the NYT10 dataset, and performs especially well at higher recall levels.}, booktitle = {Proceedings of the 57th {Annual} {Meeting} of the {Association} for {Computational} {Linguistics}}, publisher = {Association for Computational Linguistics}, author = {Alt, Christoph and Hübner, Marc and Hennig, Leonhard}, month = jul, year = {2019}, pages = {1388--1398} } - C. Alt, M. Hübner, and L. Hennig, „Improving Relation Extraction by Pre-trained Language Representations,“ in Proceedings of AKBC 2019, Amherst, Massachusetts, 2019, p. 1–18.
[BibTeX] [Abstract] [Download PDF]
Current state-of-the-art relation extraction methods typically rely on a set of lexical, syntactic, and semantic features, explicitly computed in a pre-processing step. Training feature extraction models requires additional annotated language resources, which severely restricts the applicability and portability of relation extraction to novel languages. Similarly, pre-processing introduces an additional source of error. To address these limitations, we introduce TRE, a Transformer for Relation Extraction, extending the OpenAI Generative Pre-trained Transformer [Radford et al., 2018]. Unlike previous relation extraction models, TRE uses pre-trained deep language representations instead of explicit linguistic features to inform the relation classification and combines it with the self-attentive Transformer architecture to effectively model long-range dependencies between entity mentions. TRE allows us to learn implicit linguistic features solely from plain text corpora by unsupervised pre-training, before fine-tuning the learned language representations on the relation extraction task. TRE obtains a new state-of-the-art result on the TACRED and SemEval 2010 Task 8 datasets, achieving a test F1 of 67.4 and 87.1, respectively. Furthermore, we observe a significant increase in sample efficiency. With only 20\% of the training examples, TRE matches the performance of our baselines and our model trained from scratch on 100\% of the TACRED dataset. We open-source our trained models, experiments, and source code.
@inproceedings{alt_improving_2019, address = {Amherst, Massachusetts}, title = {Improving {Relation} {Extraction} by {Pre}-trained {Language} {Representations}}, url = {https://openreview.net/forum?id=BJgrxbqp67}, abstract = {Current state-of-the-art relation extraction methods typically rely on a set of lexical, syntactic, and semantic features, explicitly computed in a pre-processing step. Training feature extraction models requires additional annotated language resources, which severely restricts the applicability and portability of relation extraction to novel languages. Similarly, pre-processing introduces an additional source of error. To address these limitations, we introduce TRE, a Transformer for Relation Extraction, extending the OpenAI Generative Pre-trained Transformer [Radford et al., 2018]. Unlike previous relation extraction models, TRE uses pre-trained deep language representations instead of explicit linguistic features to inform the relation classification and combines it with the self-attentive Transformer architecture to effectively model long-range dependencies between entity mentions. TRE allows us to learn implicit linguistic features solely from plain text corpora by unsupervised pre-training, before fine-tuning the learned language representations on the relation extraction task. TRE obtains a new state-of-the-art result on the TACRED and SemEval 2010 Task 8 datasets, achieving a test F1 of 67.4 and 87.1, respectively. Furthermore, we observe a significant increase in sample efficiency. With only 20\% of the training examples, TRE matches the performance of our baselines and our model trained from scratch on 100\% of the TACRED dataset. We open-source our trained models, experiments, and source code.}, booktitle = {Proceedings of {AKBC} 2019}, author = {Alt, Christoph and Hübner, Marc and Hennig, Leonhard}, year = {2019}, pages = {1--18} } - J. Frey, M. Hofer, D. Obraczka, J. Lehmann, and S. Hellmann, „DBpedia FlexiFusion the Best of Wikipedia \textgreater Wikidata \textgreater Your Data,“ in The Semantic Web – ISWC 2019 – 18th International Semantic Web Conference, Auckland, New Zealand, October 26-30, 2019, Proceedings, Part II, 2019, p. 96–112. doi:10.1007/978-3-030-30796-7_7
[BibTeX] [Download PDF]@inproceedings{DBLP:conf/semweb/FreyHO0H19, author = {Johannes Frey and Marvin Hofer and Daniel Obraczka and Jens Lehmann and Sebastian Hellmann}, editor = {Chiara Ghidini and Olaf Hartig and Maria Maleshkova and Vojtech Sv{\'{a}}tek and Isabel F. Cruz and Aidan Hogan and Jie Song and Maxime Lefran{\c{c}}ois and Fabien Gandon}, title = {DBpedia FlexiFusion the Best of Wikipedia {\textgreater} Wikidata {\textgreater} Your Data}, booktitle = {The Semantic Web - {ISWC} 2019 - 18th International Semantic Web Conference, Auckland, New Zealand, October 26-30, 2019, Proceedings, Part {II}}, series = {Lecture Notes in Computer Science}, volume = {11779}, pages = {96--112}, publisher = {Springer}, year = {2019}, url = {https://doi.org/10.1007/978-3-030-30796-7\_7}, doi = {10.1007/978-3-030-30796-7\_7}, timestamp = {Fri, 09 Apr 2021 18:46:51 +0200}, biburl = {https://dblp.org/rec/conf/semweb/FreyHO0H19.bib}, bibsource = {dblp computer science bibliography, https://dblp.org} } - P. Gupta, Y. Chaudhary, and H. Schütze, „BioNLP-OST 2019 RDoC Tasks: Multi-grain Neural Relevance Ranking Using Topics and Attention Based Query-Document-Sentence Interactions,“ in Proceedings of The 5th Workshop on BioNLP Open Shared Tasks, Hong Kong, China, 2019, p. 227–236. doi:10.18653/v1/D19-5730
[BibTeX] [Download PDF]@inproceedings{gupta-etal-2019-bionlp, title = "{B}io{NLP}-{OST} 2019 {RD}o{C} Tasks: Multi-grain Neural Relevance Ranking Using Topics and Attention Based Query-Document-Sentence Interactions", author = {Gupta, Pankaj and Chaudhary, Yatin and Sch{\"u}tze, Hinrich}, booktitle = "Proceedings of The 5th Workshop on BioNLP Open Shared Tasks", month = nov, year = "2019", address = "Hong Kong, China", publisher = "Association for Computational Linguistics", url = "https://www.aclweb.org/anthology/D19-5730", doi = "10.18653/v1/D19-5730", pages = "227--236" } - P. Gupta, K. Saxena, U. Yaseen, T. Runkler, and H. Schütze, „Neural Architectures for Fine-Grained Propaganda Detection in News,“ in Proceedings of the Second Workshop on Natural Language Processing for Internet Freedom: Censorship, Disinformation, and Propaganda, Hong Kong, China, 2019, p. 92–97.
[BibTeX] [Download PDF]@inproceedings{gupta-etal-2019-neural, title = "Neural Architectures for Fine-Grained Propaganda Detection in News", author = {Gupta, Pankaj and Saxena, Khushbu and Yaseen, Usama and Runkler, Thomas and Sch{\"u}tze, Hinrich}, booktitle = "Proceedings of the Second Workshop on Natural Language Processing for Internet Freedom: Censorship, Disinformation, and Propaganda", month = nov, year = "2019", address = "Hong Kong, China", publisher = "Association for Computational Linguistics", url = "https://www.aclweb.org/anthology/D19-5012", pages = "92--97" } - P. Gupta, U. Yaseen, and H. Schütze, „Linguistically Informed Relation Extraction and Neural Architectures for Nested Named Entity Recognition in BioNLP-OST 2019,“ in Proceedings of The 5th Workshop on BioNLP Open Shared Tasks, Hong Kong, China, 2019, p. 132–142.
[BibTeX] [Download PDF]@inproceedings{gupta-etal-2019-linguistically, title = "Linguistically Informed Relation Extraction and Neural Architectures for Nested Named Entity Recognition in {B}io{NLP}-{OST} 2019", author = {Gupta, Pankaj and Yaseen, Usama and Sch{\"u}tze, Hinrich}, booktitle = "Proceedings of The 5th Workshop on BioNLP Open Shared Tasks", month = nov, year = "2019", address = "Hong Kong, China", publisher = "Association for Computational Linguistics", url = "https://www.aclweb.org/anthology/D19-5720", pages = "132--142" }