Reading List
semi-supervised learning for structured variables
알 수 없는 사용자
2008. 7. 8. 15:33
Abstract
We present a new semi-supervised training procedure for conditional random fields (CRFs) that can be used to train sequence segmentors and labelers from a combination of labeled and unlabeled training data. Our approach is based on extending the minimum entropy regularization framework to the structured prediction case, yielding a training objective that combines unlabeled conditional entropy with labeled conditional likelihood. Although the training objective is no longer concave, it can still be used to improve an initial model (e.g. obtained from supervised training) by iterative ascent. We apply our new training algorithm to the problem of identifying gene and protein mentions in biological texts, and show that incorporating unlabeled data improves the performance of the supervised CRF in this case. 1Semi-supervised conditional random fields for improved sequence segmentation and labeling- Cited by 6 – Show/Hide Context – Add To MetaCart
Abstract
The problem of learning a mapping between input and structured, interdependent output variables covers sequential, spatial, and relational learning as well as predicting recursive structures. Joint feature representations of the input and output variables have paved the way to leveraging discriminative learners such as SVMs to this class of problems. We address the problem of semi-supervised learning in joint input output spaces. The cotraining approach is based on the principle of maximizing the consensus among multiple independent hypotheses; we develop this principle into a semi-supervised support vector learning algorithm for joint input output spaces and arbitrary loss functions. Experiments investigate the benefit of semisupervised structured models in terms of accuracy and F1 score. 1.Semi-supervised learning for structured output variables- Cited by 3 – Show/Hide Context – Add To MetaCart
Abstract
We present a novel, semi-supervised approach to training discriminative random fields (DRFs) that efficiently exploits labeled and unlabeled training data to achieve improved accuracy in a variety of image processing tasks. We formulate DRF training as a form of MAP estimation that combines conditional loglikelihood on labeled data, given a data-dependent prior, with a conditional entropy regularizer defined on unlabeled data. Although the training objective is no longer concave, we develop an efficient local optimization procedure that produces classifiers that are more accurate than ones based on standard supervised DRF training. We then apply our semi-supervised approach to train DRFs to segment both synthetic and real data sets, and demonstrate significant improvements over supervised DRFs in each case. 1Learning to model spatial dependency: Semi-supervised discriminative random fields- Cited by 1 – Show/Hide Context – Add To MetaCart
Abstract
We present a new unsupervised algorithm for training structured predictors that is discriminative, convex, and avoids the use of EM. The idea is to formulate an unsupervised version of structured learning methods, such as maximum margin Markov networks, that can be trained via semidefinite programming. The result is a discriminative training criterion for structured predictors (like hidden Markov models) that remains unsupervised and does not create local minima. To reduce training cost, we reformulate the training procedure to mitigate the dependence on semidefinite programming, and finally propose a heuristic procedure that avoids semidefinite programming entirely. Experimental results show that the convex discriminative procedure can produce better conditional models than conventional Baum-Welch (EM) training. 1.Discriminative unsupervised learning of structured predictors- Cited by 1 – Show/Hide Context – Add To MetaCart
Abstract
This paper explores structured multi-label transductive clustering for learning lexical part-of-speech information in sparse-data scenarios. We propose three ways of extending existing transductive clustering schemes for binary label assignment to the multi-label case. Preliminary experimental results demonstrate that appropriately defined priors on label assignment hypotheses are crucial for obtaining good performance. 12005b. Structured multi-label transductive learning- Cited by 1 – Show/Hide Context – Add To MetaCart
Abstract
We describe Hidden Semi-Markov Support Vector Machines (SHM SVMs), an extension of HM SVMs to semi-Markov chains. This allows us to predict segmentations of sequences based on segment-based features measuring properties such as the length of the segment. We propose a novel technique to partition the problem into sub-problems. The independently obtained partial solutions can then be recombined in an efficient way, which allows us to solve label sequence learning problems with several thousands of labeled sequences. We have tested our algorithm for predicting gene structures, an important problem in computational biology. Results on a well-known model organism illustrate the great potential of SHM SVMs in computational biology. 1Abstract- Show/Hide Context – Add To MetaCart
Abstract
We study the problem of learning kernel machines transductively for structured output variables. Transductive learning can be reduced to combinatorial optimization problems over all possible labelings of the unlabeled data. In order to scale transductive learning to structured variables, we transform the corresponding non-convex, combinatorial, constrained optimization problems into continuous, unconstrained optimization problems. The discrete optimization parameters are eliminated and the resulting differentiable problems can be optimized efficiently. We study the effectiveness of the generalized TSVM on multiclass classification and labelsequence learning problems empirically. 1.Transductive Support Vector Machines for Structured Variables- Show/Hide Context – Add To MetaCart
Abstract
This paper proposes a framework for semi-supervised structured output learning (SOL), specifically for sequence labeling, based on a hybrid generative and discriminative approach. We define the objective function of our hybrid model, which is written in log-linear form, by discriminatively combining discriminative structured predictor(s) with generative model(s) that incorporate unlabeled data. Then, unlabeled data is used in a generative manner to increase the sum of the discriminant functions for all outputs during the parameter estimation. Experiments on named entity recognition (CoNLL-2003) and syntactic chunking (CoNLL-2000) data show that our hybrid model significantly outperforms the stateof-the-art performance obtained with supervised SOL methods, such as conditional random fields (CRFs). 1Semi-Supervised Structured Output Learning based on a Hybrid Generative and Discriminative Approach- Show/Hide Context – Add To MetaCart
Abstract
Abstract. Label ranking studies the problem of learning a mapping from instances to rankings over a predefined set of labels. We approach this setting from a case-based perspective and propose a sophisticated k-NN framework as an alternative to previous binary decomposition techniques. It exhibits the appealing property of transparency and is based on an aggregation model which allows to incorporate a broad class of pairwise loss functions on label ranking. In addition to these conceptual advantages, we also present empirical results underscoring the merits of our approach in comparison to state-of-the-art learning methods. 1Case-based Label Ranking- Show/Hide Context – Add To MetaCart
Abstract
Abstract not found1 Contents- Show/Hide Context – Add To MetaCart
