Puneet:
They fetch one word. To predict a word that comes after that one word.. to do that what they are doing is —
If we take a word, the words around it found in a greater count, have more chances of being predicted. They vectorise each word into n numbers. Say the word king is represented with 5 numbers - -1.2, 2.4.. etc- it is initialised in this way. The word to vector algorithm is in-itself pretty amazing.
So just imagine a 5-dimensional vector ‘king’ floating around in 5-d space that is constantly updating/iterating. In the final iteration, it exists somewhere in the 5-d space. In this way, ‘man’, ‘queen’ etc are all floating around in 5-d sapce. If you remember, A vector + B vestor = C vecotor, ok?
Sultana : Yes
Puneet: so now what’s happening in this algorithm, these iterations are on the basis of two things - ‘what is the word?’ ‘What are the contextual words around it?’ In ‘I live in Delhi’, the word ‘live’ has words on the left and right- contextual words-- on the basis of which it is updating.
So now, what they did is KING - MAN + QUEEN . This will yield a vector with 5numbers. And that vecotor was ‘WOMAN’
Sultana: No way!!
(Laughter on both sides)
Puneet: In our childhood, we used to do this -
KING:MAN :: QUEEN : ?
Thats what this algorithm is doing.
This is amazing, yes. But what I find mathematically amazing.. (pause) ok this is hard to explain-///
Here is a link to some details on this word to vector algorithm.
// there were many conversations last year at the prakruti canteen in IISc. about n-dimensional space and matrices. and about the represntation of networks in matrices.
They fetch one word. To predict a word that comes after that one word.. to do that what they are doing is —
If we take a word, the words around it found in a greater count, have more chances of being predicted. They vectorise each word into n numbers. Say the word king is represented with 5 numbers - -1.2, 2.4.. etc- it is initialised in this way. The word to vector algorithm is in-itself pretty amazing.
So just imagine a 5-dimensional vector ‘king’ floating around in 5-d space that is constantly updating/iterating. In the final iteration, it exists somewhere in the 5-d space. In this way, ‘man’, ‘queen’ etc are all floating around in 5-d sapce. If you remember, A vector + B vestor = C vecotor, ok?
Sultana : Yes
Puneet: so now what’s happening in this algorithm, these iterations are on the basis of two things - ‘what is the word?’ ‘What are the contextual words around it?’ In ‘I live in Delhi’, the word ‘live’ has words on the left and right- contextual words-- on the basis of which it is updating.
So now, what they did is KING - MAN + QUEEN . This will yield a vector with 5numbers. And that vecotor was ‘WOMAN’
Sultana: No way!!
(Laughter on both sides)
Puneet: In our childhood, we used to do this -
KING:MAN :: QUEEN : ?
Thats what this algorithm is doing.
This is amazing, yes. But what I find mathematically amazing.. (pause) ok this is hard to explain-///
Here is a link to some details on this word to vector algorithm.
// there were many conversations last year at the prakruti canteen in IISc. about n-dimensional space and matrices. and about the represntation of networks in matrices.