----------------------------------------------------------
About this workshop:
This workshop, explains Michael Coen, is an effort to engender temperate, collaborative discussion of a matter that inspires hot dispute: whether machine learning helps explain how humans acquire language. In particular, says Coen, machine learning advocates believe they have evidence against Noam Chomsky’s “poverty of stimulus argument,” which in essence states that language is built into us, that “children don’t receive enough linguistic inputs to explain linguistic outputs.”
Coen, who doesn’t think much of such claims, worries about a deeper problem, that scientists have “begun to discuss engineering at the expense of science.” He describes 13-year-old Bobby Fischer’s astonishing match with a world chessmaster, where Fischer managed to look 16 moves ahead -- eliminating about 10 to the 30th board positions. We had no way to represent his thinking process then, and we don’t today, although scientists have built a machine, Deep Blue, that can topple any human chess champion. It seems there’s nothing left to say about chess, yet we know absolutely nothing about how humans play chess, says Coen. “If you’re an engineer, this may be fine, but if you’re a scientist, that’s deeply troubling.”
One problem with machine models, says Lila Gleitman, is that “they don’t try to learn what the human already knows,” and we really aren’t sure “how big a piece of the pie that is in the first place.” Gleitman distinguishes between acquiring language, and acquiring *a* language, like French or German. In her years of researching how children learn language, and specifically children who have been deprived of linguistic input entirely, Gleitman does not find a blank slate: “Children don’t just sit there; they start to make gestures.” Gleitman reviews various studies that describe a basic sequence in language acquisition that holds true regardless of specific ‘inputs.’ If researchers make models that are to be “of any interest, they ought to take into account the fact that you may not have to learn some of this.”
Gleitman has conducted simulations with adults, giving them incomplete scenes on video or paper (dropping words or substituting Lewis Carroll type doggerel) to see how we acquire the meaning of common nouns and verbs through contextual clues and inference. The more sources of evidence people get in these tests, the better they do. But such language acquisition “doesn’t scale up” to higher level categories of words,” such as “think.” Says Gleitman, “It’s crazy…to suppose there’s no biological given in a language learning situation. There’s plenty. Some of it is maybe the substance of language and some of that is about the sophisticated learning procedures themselves.” So any kind of “informative statistical modeling requires a matrix of conspiring cues, intrinsically ordered in time of appearance…Realistic models of incremental learning will incorporate what the learner brings to the task.”
筆記:
Michael Coen 's illustration:
Statements and quotes
1. Recent demonstrations of statistical learning in infants have reinvigorated the innateness versus learning debate in language acquisition.
2.Probabilistic methods are providing new explanatory approaches to fundamental cognitive science questions of how humans structure process and acquire language.
3.Probabalistic medols can account for the learning and processing of language, while maintaining the sophistication of symbolic models.
4.Recent research on unsupervised machine learning of grammar offers support for the view that knowledge of language can be achieved through general machine learning methods with a minimal set of initial settings for possible linguistic categories and rule hypotheses.
Questons:
Epiphenomenal and incidental surface features
Language specialficity
Methodological issues:
Lack of cross-validation, a fundamental component of machine learning
Errors in results
Unjustified statistical priors: "No free lunch"theorem
Ad hoc success metrics for parsing:"Bracketing"
Text corpora selection:
Highly artificial subsets of natural language
Theoretical misunderstandings:
Auxifiliary fronting"has been one of the most important test cases in language acquisition over the last few decades."(Clark and Eyraud 2006)
Examining Clark and Eyraud 2006
Learning auxiliary fronting with grammatical inference. In Proceedings of the Tenth Conference on Natural Language Learning and Proceedings of the 28th Annual meeting of the Cognitive Science Society
They propose a simple approach to "learning auxiliary fronting in polar interrogatives in English."
This is part of a much larger set of phenomena known as displacement or movement. Here, they are interested in how childrem learn to transform declarative statements with multiple auxiliary verbs into yes/no questions.
For example,
Consider the sentence:The man who is singing is walking.
Correct fronting: Is the man who is singing walking?
Incorrect fronting: Is the man who singing is walking?
Their solution is a simple notion of week substitutability.
Skipping the definitions, notation, and proofs...
Consider strings in English,a ,b,c,d.
For example, let
a=The sleepy cat is happy
b=The dog is happy.
And if
c=sleepy cat
d=dog
Then [sleepy cat=dog]
Then we say c and d are weakly substtutable, solely because they can be substituted for one another in strings a and b. The weak designation means oif they can be substituted anywhere, they can be substituted everywhere. (Namely, a case==>all cases.)
Clain: Weakly substitutability is sufficient for learning auxiliary fronting. (Thereby contradicting evidence of poverty of stimulus in the phenomenon.)
Unsupervised Macine Learning, a la Clark and
Presentation sentences:
1. The [man who is hungry] died.
2. The man ordered dinner.
3. The [man] died.
4. The man is{hungry}.
5. Is the man hungry?
6. The man is {ordering dinner}.
Output of algorithm:
Is the man who is hungry ordering dinner?
Is the man who hungry is ordering dinner?
Running the algorithm:
Is the [man]hungry?=Is the [man who is hungry] {hungry}?Leading to:
Is the man who is hungry {ordering dinner}?
The Game of the Century--Chess in Rosenwald Memorial Tornament
Donald Byrne v. Bobby Fischer, 1956 (Impossible, Byrne is losing to a 13 year old nobody.)
Among the most famous moves in chess history, requiring a 16-move lookahead searce to justify!
IBM's Deep Blue computer could search 0(10的八次方)moves/second
WOrld's top humans can search 0[10] moves/second
How human deal with 10的30次方 position
Although computational methods work, we've learned essentially nothing about how humans solve or represent this kind of problems.
Given the engineering success, we've also stopped looking.
Two faces of language acquisition:
1) Acquiring language
reserved to the humans, denied to the pets.
robust to input variation
2) Acquiring "a" language, e.g. French
reserved to the human French-dwellers, denied to the English
Sensitive in detail to the input
Constrained by 1)
A Research strategy
- Language learning under conditions of deprivation
*Variation in material speech style (Newport, Gleitman, Gleitman 1977)
*Deprivation on the sund side: Language development in the isolated deaf (Feldman, Goldin-Meadow & Gleitmanm, 1978)
*Deprivation on the meaning side: Language development in the blind ( Landau & Gleitman, 1985).
*Degraded input (later)
Convergent learning
Lexical learning as PoS
*To be sure, a photograph may show three giraffes in the veldt; but it likewise shows: a family of giraffes; and an odd number of Granny;s favorite creatures; and a number of Granny;s favorite odd creatures; and a piece of veldt that's inhabited by any of all these.--J. Fodor. 2006
Conceptual Change
Children learning language have already isolated these cohesive packages- the concrete objects...---from their surroundings, but not the relationships expressed by verbs.
Testing the information-change hypothesis: Adults with degraded input.
1. video: 6 mothers with 18-24 month old children.
2. Select for analysis 24 most frequent nouns and 24 most frequent verbs.
3. Present the to adults for identification:(excludes conceptual change)
4. Under varying informaiton conditions
Information Change
-The requirement to learn solely from observation favors early learning of concrete items.
-The noun knowledge supports acquisition of more sophisticated linguistic representaions,
--which in turn support the learning of verbs.
observation of word you use for certain situations
first noun, then verbs
No comments:
Post a Comment