Can I decode embedings to sequences using seq2seq?

[Wang-Lin-boop]

I have a naive question, if I have a pre-trained transformer model (pre-train task is a regression probelem), can I using the embedding of this model to training a decode model to decode embeddings to sequences, as like a translation task? I wonder if seq2seq can help me on this task. Thanks, anyway.

[SCUTQiaoRui]

您的邮件已收到，，，我会尽快回复的，，，