Masked Language Model

A Language Model trained by masking a word in the input and asking the model to predict it.

There is no need for labeld.

Example Architecture ![[CleanShot 2023-10-03 at 22.22.21@2x.png]]