There's one comment that we should mention with regards to babysitting embedding models.

Remember that data quality matters when training embeddings. If you have too many examples that are too easy, your system won't learn anything interesting.