Drink to me only with thine eyes,
And I will pledge with mine;
Or leave a kiss but in the cup,
And I'll not look for wine.
The thirst that from the soul doth rise
Doth ask a drink divine;
But might I of Jove's nectar sup,
I would not change for thine.
I sent thee late a rosy wreath,
Not so much honouring thee
As giving it a hope, that there
It could not withered be.
But thou thereon didst only breathe,
And sent'st it back to me;
Since when it grows, and smells, I swear,
Not of itself, but thee.
The connective word ‘But’ is pivotal. It is the moment in the poem that tells the reader that this love affair is doomed. 
 That Celia only ‘breathed’ on the wreath implies disdain or rejection. She doesn’t even trouble to touch the flowers. 
 The speed and seeming flippancy with which Celia ‘sent'st it back’ reflects her lack of feeling for the speaker. We can assume she deliberately breathed on the flowers to create the supernatural effect of leaving her scent … and does this deliberately to torment him
Song To Celia