Yes, it will generate continuations in the indicated language, and one of the presets is even a French translation mode.

Using the regular English question-answering mode preset, I just got this:

> Q: Auf welche Sprache ist diese Frage?

> A: Diese Frage ist auf Deutsch.

> Q: Em qual idioma está essa pergunta?

> A: Essa pergunta está em português.

> Q: En cual idioma está esta pregunta?

> A: Esta pregunta está en español.

> Q: Em português, qual o número que segue o cinco?

> A: O número seguinte ao cinco é o seis.

Whoa, I just asked in in Portuguese about Italian numerals and it replied correctly (in Portuguese):

> Q: Em italiano, qual o número que segue o sei?

> A: O número que segue o sei é sette.

That is "In Italian, what's the number that follows sei?" — "The number that follows sei is sette."

Apparently so. There were some notes on Twitter that included GPT-3 generated content that was a mix of German and English and folks said the German was pretty good.

Section 3.3 of the paper[0] covers this. 93% of the training data was English, and the rest other languages. (German, French and a long tail of other languages.) This was not training data specifically for translation, but a natural mix of language as it appears in some documents.

With a few prompts to explain the translation task, GPT-3 is claimed to perform well on certain translation tasks to English. (It was not as good as the state of the art in the other direction.)

[0] https://arxiv.org/abs/2005.14165