Python/Matplotlib: saving figures as .png but axis text as .pdf or .eps

Question

I am doing some Matplotlib figures with a lot of data. When saving these as .pdf-files the figures become very large and take a long time to render; however the I much prefer the .pdf-format for the axes text. My figure-saving workflow is something like this:

import matplotlib.pyplot as plt
fig, ax = plt.subplots(1, 1)
ax.plot(x, y)

fig.savefig('plot.pdf')

or

fig.savefig('plot.png', dpi=600)

Question: Is there a way to save a figure with the plot-part as a picture (.png) and the text (axis-labels etc.) as vector graphics (.pdf)?

Cheers!