Masked Language Model

A Language Model trained by masking a word in the input and asking the model to predict it.

There is no need for labeld.

Example Architecture