APPLICATION OF SUPERVISED LEARNING
DEEP LEARNING
|
Question
[CLICK ON ANY CHOICE TO KNOW THE RIGHT ANSWER]
|
Which of the following properties will a good position encoding ideally have:
|
|
Unique for all positions
|
|
|
Relative-distances are independent of absolute sequence position
|
|
|
Well-defined for arbitrary sequence lengths absolute relative
|
|
|
None of the above
|
Explanation:
Detailed explanation-1: -In the Transformer architecture, positional encoding is used to give the order context to the non-recurrent architecture of multi-head attention. Let’s unpack that sentence a bit. When the recurrent networks are fed with sequence inputs, the sequential order (ordering of time-steps) is implicitly defined by the input.
Detailed explanation-2: -Absolute position embeddings encode the absolute position of a word in the input phrase, the first word has position 1, the 50th word has position 50. Relative position embeddings encode the relative position two words have to each other, so the relative position between words 7 and 10 in a phrase would be 3.
There is 1 question to complete.