Inductive learning is essentially learning by example. The process itself ideally implies some method for drawing conclusions about previously unseen examples once learning is complete. More formally, one might state: Given a set of training examples, develop a hypothesis that is as consistent as possible with the provided data. It is worthy of note that this is an imperfect technique. As Chalmers points out, "an inductive inference with true premises [can] lead to false conclusions". The example set may be an incomplete representation of the true population, or correct but inappropriate rules may be derived which apply only to the example set.
A simple demonstration of this type of learning is to consider the following set of bit-strings (each digit can only take on the value 0 or 1), each noted as either a positive or negative example of some concept. The task is to infer from this data (or "induce") a rule to account for the given classification:
A rule one could induce from this data is that strings with an even number of 1's are "+", those with an odd number of 1's are "-". Note that this rule would indeed allow us to classify previously unseen strings (i.e. 1001 is "+").
Techniques for modeling the inductive learning process include: Quinlan's decision trees (results from information theory are used to partition data based on maximizing "information content" of a given sub-classification) , connection decision list techniques , among others. (most neural network models rely on training techniques that seek to infer a relationship from examples) and
This paper presents a method for inducing logic programs from examples that learns a new class of concepts called first-order decision lists, defined as ordered lists of clauses each ending in a cut. The method, called FOIDL, is based on FOIL (Quinlan, 1990) but employs intensional background knowledge and avoids the need for explicit negative examples. It is particularly useful for problems that involve rules with specific exceptions, such as learning the past-tense of English verbs, a task widely studied in the context of the symbolic/connectionist debate. FOIDL is able to learn concise, accurate programs for this problem from significantly fewer examples than previous methods both connectionist and symbolic
The intrinsic accuracy of an inductive problem is the accuracy achieved by exhaustive table look-up. Intrinsic accuracy is the upper bound for any inductive method. Hard concepts are concepts that have high intrinsic accuracy, but which cannot be learned effectively with traditional inductive methods. To learn hard concepts, we must use constructive induction - methods that create new features. We use measures of concept dispersion to explore (conceptually and empirically) the inherent weaknesses of traditional inductive approaches. These structural defects are buried in the design of the algorithms and prevent the learning of hard concepts. After studying some examples of successful and unsuccessful feature construction ("success" being defined here in terms of accuracy), we introduce a single measure of inductive difficulty that we call variation. We argue for a specific approach to constructive induction that reduces variation by incorporating various kinds of domain knowledge. All of these kinds of domain knowledge boil down to utility invariants, i.e., transformations that group together non-contiguous portions of feature space having similar class-membership values. Utility invariants manifest themselves in various ways: in some cases they exist in the user's stock of domain knowledge, in other cases they may be discovered via methods we describe
